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Three  hundred  and  sixty  simulated  experiments  were  conducted 
to  compare  Type  I  error  rates  of  the  Pillai-Bartlett  trace, 
Johansen's  test,  James'  first  order  test,  and  James'  second  order 
test  when  the  covariance  matrices  are  heterogeneous.  Both  equal  and 
unequal  sample-size  conditions  were  investigated. 

Results  of  this  study  indicated  that  the  ratio  of  total  sample 
size  to  the  number  of  variables  had  a  strong  impact  on  the 
performance  of  the  tests.  As  the  ratio  increased,  the  tests  become 
more  robust,  except  the  Pillai-Bartlett  trace.  With  one  exception, 
as  the  covariance  heteroscedasticity  increased,  the  performance  of 
the  tests  declined.  The  exception  was  James'  second  order  test, 
which  performed  better  when  heteroscedasticity  increased  and  the 
ratio  of  total  sample  size  to  the  number  of  variables  was  20.   When 


the  smaller  samples  were  associated  with  the  covariance  matrices 
with  smaller  elements,  the  performance  of  Johansen's  test  and  James' 
first  order  test  improved.  When  the  smaller  samples  were  associated 
with  the  covariance  matrices  with  larger  elements,  the  performance 
of  James'  second  order  test  improved.  The  number  of  variables 
impacted  only  James'  second  order  test.  The  performance  of  the  test 
declined  as  the  number  of  variables  increased.  The  sample  size 
ratio  affected  the  tests  only  when  the  ratio  of  the  total  sample 
size  to  the  number  of  variables  was  small.  The  form  of  covariance 
matrices  affected  only  the  performance  of  the  Pillai-Bartlett  trace. 
In  conclusion,  Johansen's  test  and  James'  second  order  test 
performed  better  than  the  Pillai-Bartlett  trace  and  James's  first 
order  test  when  covariance  matrices  were  unequal.  Johansen's  test 
tended  to  have  estimated  Type  I  error  rates  larger  than  a  and  James' 
second  order  test  tended  to  have  estimated  Type  I  error  rates 
smaller  than  a.  However,  for  both  tests  the  Type  I  error  rates  were 
reasonably  near  a. 


CHAPTER  1 
INTRODUCTION 

It  is  common  in  educational  research  to  compare  two  or  more 

samples  on  several  criterion  variables.    That  is,  p  response 

variables  are  observed  in  k  treatment  groups  of  n,  experimental  units 

per  group  (i  =  1,   ...,  k)  .    Multivariate  analysis  of  variance 

(MANOVA)  can  be  employed  to  analyze  the  data.   When  p  -  1,  the 

univariate  F  test  can  be  used.   When  the  assumptions  of  the  F  test 

are  met,  it  is  the  uniformly  most  powerful,  invariant  test  of  the 

null  hypothesis  that  the  means  of  the  populations  from  which  the  k 

samples  are  drawn  are  equal.   However,  in  MANOVA  there  is  no  single 

invariant  test  that  is  uniformly  most  powerful.   Four  MANOVA  test 

criteria  are  commonly  employed: 

1.  Roy's  largest  root  (R) ; 

2.  The  Hotelling-Lawley  trace  (T) ; 

3.  Wilks'  likelihood  ratio  (W) ; 

4.  The  Pillai-Bartlett  trace  (V). 

When  either  k  -  2  or  p  =  1 ,  the  four  test  criteria  lead  to  the  same 
decision  about  the  null  hypothesis. 

One  of  the  major  assumptions  of  MANOVA  is  homogeneity  of  the 
covariance  matrices  in  the  k  populations  from  which  the  samples  are 
drawn.  Stevens  (1986)  pointed  out  that  the  restric tiveness  of  the 


2 

assumption  becomes  more  strikingly  apparent  when  we  realize  that  the 
corresponding  assumption  for  the  univariate  ANOVA  is  that  the 
variances  on  only  a  single  variable  are  equal.  Therefore,  it  is 
very  unlikely  that  the  equal  covariance  matrices  assumption  ever 
would  be  absolutely  satisfied  in  practice. 

Olson  (1974)  provided  a  general  picture  of  the  behavior  of 
the  MANOVA  criteria  under  violation  of  the  covariance  homogeneity 
assumption.  According  to  Olson,  for  protection  against  covariance 
heterogeneity  in  the  population,  the  T,  W,  and  R  tests  should  be 
avoided.  Although  the  V  test  stands  up  best  to  violations  of 
homogeneity  of  covariance  matrices,  its  actual  Type  I  error  (r)  is 
still  somewhat  high.  For  example,  under  various  simulated 
experimental  conditions,  Olson  (1973)  reported  that  the  estimated 
Type  I  error  rate,  r,  of  the  V  test  ranged  from  .061  to  .186  when 
the  nominal  rate  was  a  =  .05. 

James  (1954)  and  Johansen  (1980)  proposed  multivariate  tests 
for  the  situation  in  which  the  covariance  homogeneity  assumption  is 
violated.  In  their  tests,  the  covariance  matrices  of  k  populations 
are  estimated  separately.  Therefore,  the  common  covariance  matrix 
assumption  is  not  required  for  the  tests  that  are  solutions  to  the 
multivariate  Behrens-Fisher  problem. 

The  univariate  Behrens-Fisher  problem  is  that  of  testing 
equality  of  the  means  of  two  populations  without  assuming  that  the 


3 
population  variances  are  equal.  According  to  Yao  (1965),  Behrens 
was  the  first  to  offer  a  solution  to  the  Behrens- Fisher  problem. 
Fisher  (1935,  1939)  showed  that  Behrens'  solution  could  be  derived 
by  using  his  theory  of  statistical  inference  called  "fiducial 
probability .  " 

Welch  (1938)  proposed  an  approximate  degrees  of  freedom  (APDF) 
solution  to  the  Behrens -Fisher  problem,  a  solution  in  which  the  test 
statistic  is  approximately  distributed  as  Student's  t  with 
appropriately  adjusted  degrees  of  freedom.  Later,  Welch  (1951) 
proposed  an  APDF  solution  for  the  k-sample  case,  a  solution  in  which 
the  distribution  of  the  test  statistic  is  approximated  by  an  F 
distribution  with  adjusted  degrees  of  freedom.  Welch  (1947) 
proposed  a  series  solution  to  two-sample  Behrens -Fisher  problem  and 
presented  the  solution  to  terms  of  order  1/f*,  where  f,  -  n,  -  1. 
Aspin  (1948)  extended  Welch's  approximation  to  the  terms  of  order 
1/f*.  James  (1951)  proposed  a  solution  to  the  k-sample 
Behrens -Fisher  problem.  He  also  used  the  series  expansion 
technique.  The  distribution  of  his  test  statistic  is  approximated 
by  a  function  of  the  x2   distribution. 

Welch's  (1938,  1951)  APDF  solution  was  generalized  by  Yao 
(1965)  to  solve  the  multivariate  two-sample  Behrens -  Fisher  problem. 
It  was  further  generalized  by  Johansen  (1980)  to  solve  the 
multivariate  k-sample  Behrens-Fisher  problem.   James'  (1951)  series 


solution  was  generalized  to  the  multivariate  k-sample  situation  by 
James  (1954),  who  reported  both  a  first  order  and  a  second  order 
test . 

The  Problem 
The  behavior  of  Welch's  APDF  test  and  James'  series  test  has 
been  investigated  by  several  authors  (Brown  &  Forsyth,  1974;  Wang, 
1971).  The  APDF  solution  in  the  two-sample  multivariate  situation 
has  also  been  studied  (Yao,  1965;  Algina  &  Tang,  1988).  However, 
the  behavior  of  Johansen's  APDF  solution  and  James'  series  solution 
to  multivariate  k-sample  Behrens-Fisher  problem  is  an  unexplored 
topic . 

The  researches  listed  in  the  preceding  paragraph  demonstrated 
that  with  heteroscedastic  covariance  matrices  (variances  in  the 
univariate  case),  the  APDF  and  series  solutions  to  both  the 
univariate  and  multivariate  Behrens-Fisher  problems  performed  better 
than  the  usual  tests,  such  as  t,  F,  and  Hotelling's  T2,  for  which 
homoscedasticity  is  assumed.  Furthermore,  the  studies  demonstrated 
that  the  APDF  tests  are  superior  to  James'  first  order  tests  in 
controlling  r.  Whether  this  is  still  true  when  the  tests  are 
applied  in  the  multivariate  k-sample  situation  is  unknown. 

Dijkstra  and  Werter  (1981),  on  the  other  hand,  showed  that 
James'  second  order  test  gives  better  control  of  r  than  does  Welch's 
APDF  test  in  the  univariate  k-sample  case.   Their  findings  suggest 


5 
that  the  superiority  of  Welch's  APDF  test  to  James'  first  order  test 
may  be  due  to  omitting  the  second  order  term  in  James'  test.  With 
modern  computer  facilities  it  may  be  preferable  to  include  James' 
second  order  term  in  the  multivariate  test. 

The  behavior  of  MANOVA  has  been  studied  by  Olson  (1974).  He 
concluded  that  among  the  four  criteria  only  the  V  test  is  relatively 
robust  against  violation  of  the  equal  covariance  matrices 
assumption.  However,  his  study  was  restricted  to  an  equal  number 
of  experimental  units  in  each  of  k  samples.  Therefore, 
investigating  the  performance  of  James'  and  Johansen's  tests  when 
the  sample  sizes  are  unequal  and  comparing  them  to  the  performance 
of  V  under  the  same  condition  is  merited. 

Purpose  of  the  Study 

The  purpose  of  the  present  study  was  to  compare  Type  I  error 
rates  of  the  Pillai-Bartlett  trace  (V),  Johansen's  test,  and  James' 
first  and  second  order  tests  when  the  covariance  matrices  are 
heterogeneous.  Both  equal  and  unequal  sample-size  conditions  were 
investigated. 

Significance  of  the  Study 

The  application  of  multivariate  analysis  of  variance  (MANOVA) 
in  education  and  the  behavioral  sciences  has  increased  dramatically, 
and  it  appears  that  it  will  be  used  frequently  for  data  analysis  in 
the  future  (Bray  &  Maxwell,  1985).   Stevens  (1986)  provided  three 


reasons  why  multivariate  analysis  is  necessary: 

1.  Any  worthwhile  treatment  will  affect  the  subject  in  more 
than  one  way.  Hence  the  problem  for  the  investigator 
is  to  determine  in  which  specific  ways  the  subjects  will 
be  affected,  and  then  find  sensitive  measurement 
techniques  for  those  variables. 

2.  Through  the  use  of  multiple  criterion  measures  we  can 
obtain  a  more  complete  and  detailed  description  of  the 
phenomenon  under  investigation. . . . 

3.  Treatments  can  be  expensive  to  implement,  while  the  cost 
of  obtaining  data  on  several  dependent  variables  is 
relatively  small,  and  maximizes  information  gain.  (p.  2) 


Because  the  assumption  of  equal  covariance  matrices  of  k 
populations  is  unlikely  to  be  satisfied  in  practice,  there  is 
potential  danger  that  r  will  not  be  equal  to  a  when  using  the  usual 
MANOVA  criteria.  In  this  study,  estimates  of  r  were  reported  for 
four  tests  that  can  be  employed  in  MANOVA:  Pillai-Bartlett  trace, 
James'  first  order  test,  James'  second  order  test,  and  Johansen's 
test.  Both  equal  and  unequal  sample  sizes  were  employed.  The 
results  provided  evidence  about  whether  or  not  the  latter  three 
tests  are  good  alternatives  to  the  Pillai-Bartlett  trace,  which  was 
reported  by  Olson  to  be  more  robust  when  sample  sizes  are  equal  than 
Roy's,  Wilk's,  or  the  Lawley-Hotelling  test. 


CHAPTER  2 
REVIEW  OF  LITERATURE 


The  Behrens-Fisher  Problem 

One  of  the  most  common  situations  in  educational  research  is 
the  comparison  of  the  means  of  two  populations.  If  we  assume  the 
variable  is  normally  distributed  in  each  population  and  has  equal 
variances  in  the  two  populations,  the  t  test  is  the  uniformly  most 
powerful  test  for  the  hypothesis  that  the  means  are  equal.  The 
robustness  of  t  when  the  assumption  of  variance  equality  is  violated 
has  been  studied  extensively.  Scheffe  (1959)  summarized  results  of 
studies  by  Hsu  (1938)  and  Welch  (1938,  1951).  In  general,  if  the 
two  samples  are  of  equal  size,  the  distribution  of  t  is  changed  very 
little  by  inequality  of  variance,  especially  for  large  sample  sizes. 
However,  if  the  sample  sizes  are  unequal,  inequality  of  the 
variances  affects  the  distribution  considerably.  The  t  test  is 
liberal  when  the  larger  sample  is  drawn  from  the  population  with 
smaller  variance.   It  is  conservative  otherwise. 

Yao  (1965)  pointed  out  that  Behrens  was  the  first  to  offer  a 
solution  to  the  problem  of  testing  equality  of  the  means  of  two 
populations  without  assuming  that  the  variances  are  equal.  This 
problem  has  become  known  as  the  Behrens-Fisher  problem.    Fisher 
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(1935,  1939)  showed  that  Behrens '  solution  could  be  derived  using 
his  theory  of  statistical  inference  called  "fiducial  probability." 
Fisher  and  Healy  (1956)  calculated  tables  of  the  Behrens- Fisher 
distribution.  James  (1959)  and  Ray  and  Pitman  (1961)  further 
studied  the  distribution. 

Six  other  solutions  to  the  Behrens-Fisher  problem  were 
discussed  and  compared  by  Scheffe  (1970).  According  to  Scheffe, 
within  the  Neyman-Pearson  school  of  thought,  Welch's  approximate 
degrees  of  freedom  solution  (APDF) ,  which  requires  only  the 
ubiquitous  t-table,  is  a  satisfactory  practical  solution  to  the 
Behrens-Fisher  problem. 

Welch's  Solutions 
APDF  Solution 

Let 


x, 

-    X- 

S2 

n1 

-  +  - 

s? 

^ 

where  X{  is  the  mean  of  sample  drawn  from  population  i,  and  S2  is  the 
sample  variance.  The  distribution  of  v  depends  to  a  certain  extent 
on  a\/a\.  Welch  (1938)  approximated  the  distribution  of  S^/n,  +  S\/t^ 
by  the  \2  distribution  such  that  the  mean  and  variance  of  the  \2 
distribution  agree  with  the  true  mean  and  variance  of  S^/n,  +  S^/rij. 
This  enabled  Welch  to  approximate  the  distribution  of  u  by  a  t 
distribution  with  degrees  of  freedom 


f  = 


fjL  +  JLV 


n?(n,  -  1)    n|(n2  -  1) 

Welch's  approximation  involves  referring  1/  to  a  table  of  the  t 

distribution,  entered  with  f  degrees  of  freedom.    Welch  (1947) 

showed  that  the  errors  involved  in  making  probability  statements 

about  v   on  the  basis  of  this  particular  approximation  are  of  order 

1/f,. 

Series  Solution 

In  addition  to  his  APDF  solution,  Welch  (1947)  derived  an 
exact  solution,  in  the  sense  that  it  is  independent  of  the 
irrelevant  population  parameters  a\.  Let  X,  and  X2,  the  means  of  two 
samples,  be  distributed  independently  and  normally  with  means  /x,  ,  fi2 
and  variances  crf/n, ,  o\/^  respectively.  Suppose  that  the  observed 
data  provide  estimates  S2t  of  the  variances,  a\,  and  these  estimates 
are  distributed  independently  of  each  other  and  of  X,  -  Xg.   Then, 

Pr  [(X,  -  \)    -  (/x,,  M2)  <  h(S?,  S|,  P)]  =  P 
where 

0  <  P  <  1. 

The  quantity  h(S*,  S| ,  P)  is  a  function  of  S^  and  P  such  that  P  is 
the  probability  that  (X,  -  X\>)  -  (/x,  -  M2)  is  smaller  than  h(S*,  S|, 
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P).  To  test  the  hypothesis  HQ:  /i,  =  m2,  h(S2,  S| ,  P)  is  calculated, 
and  serves  as  a  critical  value  for  comparing  the  population  mean 
differences.  Welch  (1947)  provided  a  formula  for  h(S2,  S|,  P)  up  to 
terms  of  order  1/f2: 


h(sf,  s|,  p)  =  aw2)' 


(i  +  e) 


r   a2s4  l 


(SA.S?)' 


(15   +    32C2  +   9|4  ) 
32 


(S.A.Sf)* 


(3  +  se2  +  ?4  ) 


AfS« 


A3Sb 
E,  2JLTJ 


(S,A,S2) 


2^3 

l"  i  ■ 


(Wfr 


where  £  is  the  normal  deviate  such  that  *(f)  =  P,  and  A,  =  1/n,. 
Aspin  (1948)  extended  the  work  further  to  the  terms  of  order  1/f , . 
In  addition,  tables  of  critical  values  were  computed  by  Aspin  (1948, 
1949)  and  Trichett,  James,  and  Welch  (1954,  1956).  The  limiting 
behavior  of  the  series  solution  was  studied  by  Wald  (1955) .  Welch's 
series  solution  to  the  Behrens-Fisher  problem  agrees  with  Welch's 
(1938)  APDF  solution  up  to  and  including  the  term  of  order  1/f,. 
James'  Series  Solution  for  Comparing  Several  Means 
James  (1951)  derived,  for  the  k-sample  case,  a  series  solution 
to  the  Behrens-Fisher  problem.   Let  X,,   .  .  .  ,  Xk ,  the  means  of  k 
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samples,  be  distributed  independently  and  normally  with  means  /*, , 
...,  Mk  and  variances  c^/n, ,  ...,  ^/nk,  respectively.  Let  S2  (i  = 
l,...,k)   be   estimates   of   the    a\.  The   S2  are   distributed 

independently  of  the  X,  and  of  each  other.  To  test  the  hypothesis 
H0  :  p.   -  n2   -  ...  -  Mk,  we  calculate  the  statistic 


k 

I     H  (X,  -  X)2 
i  =  1 


where 


and 


W,  =  (S2  /  n,)"1 


k 
W  -     I   W, 
i  =  1 


k 

I  W,  X, 

i  -  1 


W 


If  the  n,  are  all  large,  £,wi  (X,  -  X)2  can  be  taken  to  have  the  \2 
distribution  with  k  -  1  degrees  of  freedom.  However,  if  the  n,  are 
not  all  large,  the  test  statistic  may  not  have  a  \2  distribution 
with  k  -  1  degrees  of  freedom.   James  (1951)  sought  to  derive  a 
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series  function  of  the  S2  such  that 

Pr  [  I,W,  (X,  -  X)2  <  2h(S2)]  =  P 
James  found  approximations  to  2h(S2)  of  orders  1/f,  and  1/f , .   To 
order  of  1/f.,  he  found 


2h(S2)  =  xUa 


1  + 


3xli.a  +  (k  +  D 
2(k2  -  1) 


The  quantity  2h(S2)  serves  as  James  critical  value.  The  test 
statistic,  I.W^X,  -  X)2,  is  compared  with  this  critical  value  and 
H0  is  rejected  if  ^(X,  -  X)2  >  2h(S2)  .  James  also  provided  the 
approximation  of  order  1/f2,  which  can  be  shown  to  agree  with 
Welch's  (1947)  result  when  k  =  2.  However,  James  pointed  out  that 
this  second  order  approximation  involves  a  good  deal  of  numerical 
calculation. 

Welch's  APDF  Solution  for  Comparing  Several  Means 
Welch  derived  an  alternative  series  solution  for  the  k-sample 
case.   By  comparing  the  cumulant- generating  function  of  XjW^X,  -  X) 
with  that  of  the  F  distribution,  Welch  brought  the  two  into 
agreement.    Welch   found   that   to  order   1/f,,   I.W^X,   -   X)   is 
distributed  as  C  times  F,  where 


c  -  k  -  i  +  2(k  -  2)  z,  -1 

k  +  1      f, 
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This  F  distribution  has  degrees  of  freedom  df,  =  k  -  1  and 


df, 


f    3   *■  l 

1 

-   ^L 

L(k2   -    1)          f, 

. 

W 

The  steps  in  the  approximate  test  procedure  are  calculating 


Z,W,(X,  -  X)2  /  (k  -  1) 


i  +    2(k  -  2)  I,  — i 


<k'  -  i) 


and  referring  u2   to  F  table  entered  with  degrees  of  freedom  df,  and 
df,. 


Behavior  of  Welch's  and  James'  Tests 
Wang  (1971)  compared  the  Behrens -Fisher  (Fisher,  1935,  1939) 
test  and  the  Welch  (1938)  APDF  test  for  two  samples.  The  results 
show  that  at  n,  -  7  and  n2  =  13,  the  maximum  deviation  from  the 
nominal  a  level  is  0.0144  for  the  Behrens-Fisher  test  and  0.0017  for 
Welch's  APDF  test.  Combining  all  of  the  experimental  conditions, 
Wang  found  that  the  maximum  deviation  of  Welch's  APDF  test  from 
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nominal  a  level  is  about  0.001.  For  larger  sample  sizes  in  both 
samples,  such  as  13:19,  the  maximum  deviation  from  the  nominal  a 
level  decreases  to  0.0005.  Wang  also  found  that  Welch-Aspin  series 
solution  (Welch  1947 ;  Aspin,  1948)  gives  even  smaller  deviations 
from  the  nominal  a  than  Welch's  APDF  test.  However,  Wang  pointed 
out  that  because  the  Welch-Aspin  critical  values  are  available  for 
only  a  selected  set  of  a,  f,  and  f  2 ,  and  the  actual  computation  for 
the  critical  values  is  very  tedious,  it  seems  reasonable  to  use  the 
Welch  APDF  test,  which  requires  only  the  t-table.  Scheffe  (1970) 
also  opined,  as  mentioned  earlier,  that  Welch's  APDF  solution  is  a 
satisfactory  practical  solution  of  the  Behrens- Fisher  problem. 

Brown  and  Forsyth  (1974)  reported  that  in  the  k-sample 
situation,  r  for  the  ANOVA  F  test  deviates  markedly  from  the  nominal 
a  level  when  the  variances  of  the  populations  are  unequal.  The  Type 
I  error  rates  of  Welch's  (1951)  APDF  solution  and  James'  (1951) 
solution,  on  the  other  hand,  are  near  the  nominal  a  levels.  They 
further  pointed  out  that  Welch's  (1951)  APDF  solution  is  superior 
to  that  of  James  (1951)  series  solution  in  controlling  r,  especially 
when  the  sample  sizes  are  small.  Dijkstra  and  Werter  (1981)  showed 
that  James'  second  order  test  gives  better  control  of  r  than  does 
Welch's  test.   Their  conclusions  were  supported  by  Wilcox  (1987). 
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Hotelling's  Two  Sample  T2 
Suppose  the  purpose  is  to  compare  the  mean  vectors  p,  and  n2 
of  two  p-dimensional  multivariate  normal  populations  on  the  basis 
of  a  random  sample  from  each.  The  two  populations  are  assumed  to 
have  the  same,  though  unknown,  covariance  matrix  £  of  full  rank  p. 
Let  the  samples  be  denoted  by 

(x(|,  j  -  1,  2,  ....  ry,  I  -  1,  2) 
Define  the  sample  mean  vectors  and  covariance  matrices  as 


i-    i 

"'  3-1 


s, 


-  -h4-i      z'  (x"  "  x,Kx"  "  Xl)' 

J-l 


For  testing  the  hypothesis  HQ:  /i,  -  n2  vs  H, :  /i,  *  /i2,  Hotelling 
(1931)  proposed  the  T2  statistic  which  is  a  generalization  of 
univariate  t  test.   The  test  statistic  is 

T2  .    ",  "?     (x,  -  x2)'S"1(x,  -  x2) 


where 


(n,  -  1)S1  +  (il,  -  1)S2 
n,  +  r^   -    2 
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Hotelling  showed  that  the  quantity 


a,  +  ix,  -  p  -  1   T2 

(n,  +  r^  -  2)p 


has  an  F  distribution  with  degrees  of  freedom  p  and 
n.  +  ru  -  p  -  1.   H0  is  rejected  if 


r2  >   ("1  +  "2  -  2>P   F 

S;  +  nj  -  p  -  i   a'P' 


where  u   =  n,  +  rig  -  p  -  1 . 

Behavior  of  T2  When  Covariance  Matrices  Are  Unequal 
The  robustness  of  Hotelling' s  T2  to  the  violation  of 
homogeneous  covariance  matrices  assumption  has  been  investigated  by 
many  researchers  both  analytically  (Ito  &  Schull,  1964)  and 
empirically  (Algina  &  Oshiina ,  1989;  Hakstian,  Roed,  &  Lind,  1979; 
Holloway  &  Dunn,  1967;  Hopkins  &  Clay,  1963).  The  inequality  of 
covariance  matrices,  S,  *  Y~,,  is  reflected  by  the  eigenvalues  0,, 
i  -  1 ,  2,  ...,p,  of  the  matrix  S^E^1 . 

Ito  and  Schull  (1964)  investigated  the  large-sample  properties 
of  the  T2  statistic  in  the  presence  of  unequal  covariance  matrices 
S,  and  Ej.   They  demonstrated  that  when  n,  and  ru,  are  equal  and  large, 
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and  the  eigenvalues  of  S,^1  are  equal  to  one  another  but  are  not 
equal  to  one,  heteroscedasticity  has  no  effect  upon  r.  Ito  and 
Schull  also  investigated  the  situation  in  which  the  eigenvalues  of 
E^"1  are  distinct  and  both  samples  are  very  large,  but  only  for  p 
=  2.  They  reported  r  near  a  provided  the  eigenvalues  are  within  the 
range  ( . 5 ,  2)  . 

Hakstian,  Roed,  and  Lind  (1971)  also  found  that  with  equal 
sample  sizes  the  T2  procedure  is  generally  robust  with  respect  to 
violation  of  the  homogeneity  of  covariance  matrices  assumption. 
However,  even  when  n,  =  r^,  other  factors,  such  as  sample  sizes,  p, 
and  8,  also  play  important  roles  in  the  robustness  of  T2.  Based  on 
1000  replications,  Hopkins  and  Clay  (1963)  showed  that  when  a,  -  r^ 
>  10,  T2  is  rather  robust  against  variance  inequality  but  that  this 
robustness  does  not  extend  to  smaller  sample  sizes.  For  example, 
when  n,  -  rb,  =  5,  p  -  2,  0,  -  62  -  10.24  and  a  -  0.05,  r  was  .083. 
Holloway  and  Dunn  (1967)  found  that  for  large  and  equal  sample 
sizes,  the  robustness  of  T2  depends  on  p.  For  example,  when  n,  =  r^ 
-  50,  0,  «  S2  =  10,  r  is  close  to  a  for  p  =  2  and  p  -  3,  but  departs 
fairly  markedly  from  a  when  p  =  7  or  p  *=  10. 

For  unequal  sample  sizes,  in  all  of  the  preceding  studies, 
researchers  found  that  the  test  leads  to  unacceptable  Type  1  error 
rates  as  the  degree  of  population  covariance  matrix  heterogeneity 
is  increased.   Even  for  p  =  2  and  relatively  mild  heterogeneity,  for 
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example,  5^  =  2. 252,,  a  sample  size  ratio  of  only  1.1:1  can  produce 
an  unacceptable  Type  I  error  rate  (Algina  &  Oshima,  1989).  When  the 
larger  sample  is  drawn  from  the  population  with  the  smaller 
covariance  matrix,  the  test  results  in  overestimation  of 
significance.  When  the  larger  sample  is  drawn  from  the  population 
with  the  larger  covariance  matrix,  the  test  results  in  under- 
estimation of  significance.  This  tendency  increases  with  the 
magnitude  of  the  inequality  of  two  samples,  with  the  degree  of 
heterogeneity,  and  with  p. 

In  general,  with  large  and  equal  sample  sizes  and  a  fairly 
large  ratio  of  (n,  +  rt,)  to  p ,  the  T2  test  is  robust.  Otherwise,  the 
test  is  less  robust,  being  conservative  when  the  larger  sample  is 
drawn  from  the  population  with  generally  greater  dispersion,  and 
liberal  in  the  opposite  situation.  Therefore,  the  behavior  of  T 
under  violation  of  homogeneous  covariance  matrices  assumption  is 
similar  to  that  of  the  univariate  t  test. 

Yao's  and  James'  Tests 

James  (1954)  and  Yao  (1965)  proposed  tests  to  deal  with 
multivariate  Behrens -Fisher  problem  when  there  are  two  samples. 
The  tests  allow  the  assumption  of  equal  population  covariance 
matrices  to  be  relaxed.  James'  (1954)  test  is  a  generalization  of 
James'  (1951)  univariate  solution  of  the  Behrens- Fisher  problem. 
Yao's   test   is  a  generalization  of  Welch's   (1938,   1951)  APDF 
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solution.       The    test    statistic    is 


T2  =    (x,    -    x2)'(S1/n1   +   S2/n2)1(x1    -    x2)       . 

This  test  statistic  is  asymptotically  distributed  as  chi-square  with 
p  degrees  of  freedom. 

James  proposed  a  critical  value,  2h(S2) ,  which  is  a 
generalization  of  his  univariate  critical  value  discussed  in  the 
previous  section.   To  order  1/f,,  this  critical  value  is 


2h(S2)  =  xl    (A  +  Exi) 


where 


A  =  1  +  2  £ 


tr  (I  -  V1  W.) 


1 


P(p  +  2) 


tr  (I  -  V1  WV 


1 

Ir 

tr 

(I 

- 

W1 

w,)]2l 

2 

f, 

In  A  and  B,  W,  and  W  are  defined  in  the  same  manner  these  quantities 
are  defined  in  James'  univariate  test,  but  with  sample  covariance 
matrices  replacing  sample  variances. 

Yao  (1965)  generalized  Welch's  (1938,  1951)  solution  to  the 
Behrens-Fisher  problem.  His  critical  region  is  based  on  Hotelling's 
T2   with  p  and  f  degrees  of  freedom.   Let  y  =  x,  -  x2  -  fi0,    A,  - 
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S./a,  and  A  =  A,  +  Ag.  Define  V,  =  y'A1AA"1y-  The  approximate  degrees 
of  freedom  (APDF)  fT  is  defined  by 


fT      1-1    f. 


Therefore,  the  critical  region  is 


T$  >  T2  (p,  fT) 


If  tables  of  T2  are  not  available,  the  transformation 


F  -    fT  -  P  +  l 


T^7 


can  be  used.  Thus  we  reject  HQ:  /*i  *  M2  If  Fv  >  F(P>  ^t  "  P  +  *■)  ■ 
Behavior  of  Yao's  and  James'  Tests 
Yao  (1965)  conducted  a  simulation  study  to  estimate  r  for 
James'  test  and  his  own  test.  The  results  indicated  that  both  tests 
are  quite  robust.  However,  the  generality  of  this  conclusion  is 
limited  because  Yao  only  studied  p  =  2.  Ito  (1969)  investigated  the 
behavior  of  James'  test  analytically.   He  studied  both  equal  and 
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unequal  sample  sizes.  For  k  =  2,  the  sample  sizes,  n,  +  r^,  were  20, 
300,  and  600.  He  found  that  when  rr,  +  r^  =  300  or  600,  the  test  was 
quite  robust.  When  the  total  sample  size  was  20,  the  test  was 
liberal;  i.e.,  r  is  greater  than  a.  This  tendency  increases  as 
sample  sizes  decreases,  the  inequality  of  sample  sizes  increases, 
the  inequality  of  covariance  matrices  increases,  and  p  increases. 
The  behavior  of  James'  test  for  sample  sizes  between  20  and  300  was 
not  included  in  Ito's  study. 

Algina  and  Tang  (1988)  investigated  the  performance  of  T2, 
Yao's  test,  and  James'  test  under  a  broader  set  of  conditions  than 
Yao  (1965)  studied.  In  the  Algina-Tang  study,  p  =  2,  6,  or  10;  (n, 
+  ngVp  -  6,  10,  or  20;  n,  :  r^  ranged  from  1:1.25  to  1:5;  and  6t 
ranged  from  1.5  to  3.0.  They  found  that  the  deviation  of  r  from  a 
is  larger  for  Hotelling's  T2  than  for  either  Yao's  or  James'  test 
when  6*1.  Neither  Yao's  nor  James'  test  tends  to  be  conservative. 
The  t  for  James'  test  tends  to  be  larger  than  that  for  Yao's  test. 
Algina  and  Tang  concluded  that  in  terms  of  control  of  Type  I  error 
rates,  Yao's  test  is  preferable  to  James'  test.  When  the  sample 
sizes  are  equal  (n,  -  n2  =  n) ,  Yao's  test  has  appropriate  Type  I 
error  rates  provided  (2n/p)  >  10.  When  n1  *  n2  and  10  <  (n,  +  r^ 
)/p,  Yao's  test  can  also  be  safely  used,  provided  the  ratio  of  the 
larger  to  the  smaller  sample  size  is  less  or  equal  to  2:1.  When  n, 
*   r^  and  (n,  +  n2)/p  >  20,  Yao's  test  can  be  safely  used  a)  if  p  =  2 
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and  the  ratio  of  the  two  samples  is  5:1  or  smaller;  b)  if  p  -  6  and 
the  ratio  is  3:1  or  smaller;  and  c)  if  p=10  and  the  ratio  is  4:1  or 
smaller . 

MANOVA 
Suppose  the  purpose  is  to  compare  the  means  of  /j,  ,  n2 ,  .  .  .  ,  ,  Mk 
of  k  p-dimensional  normal  populations  on  the  basis  of  a  random 
sample  from  each.  Let  H(p  x  p)  and  E(p  x  p)  be  the  sums -of -products 
matrices  for  hypothesis  and  error,  respectively,  defined  in  the 
one-way  case  as 


k 
H  -  I       n,  (x,  -  x)(x,  -  x)' 
i-1 


k   n, 

E  =    I   I   (x„  -  X.XX,,  -  x,)' 
i-1  j=l 

where  x  is  the  i th  observation  vector  in  group  i,  x  is  the  mean 

ii        ~> 

vector  for  the  ith  group  and  x  is  the  grand  mean  vector.  It  is 
known  that  all  invariant  test  criteria  are  functions  of  the 
eigenvalues  of  HE"1  (the  rank  of  which  is  s  =  min(p,  k  -  1)).  There 
are  four  popular  test  statistics,  each  a  different  function  of  the 
eigenvalues  of  HE"1.  Let  Am  be  the  mth  eigenvalue  of  HE"1,  m  =  1, 
...,  s.   The  four  test  criteria  are: 

1.   Roy's  (1945,  1953)  largest  root  (R)  .   R  =  A,/(l  +  A,), 
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where  A,  is  the  largest  eigenvalue  of  HE1; 

2.  The  Hotelling-Lawley  (Lawley,  1938,  1939;  Hotelling,  1951) 
trace  (T) .   T  =  XmAm ; 

3.  Wilk's  (1932)  likelihood  ratio(W).   W  =  \\ml/d   +  Am)  ; 

4.  The  Pillai-Bartlett  (Bartlett,  1939;  Pillai,  1955)  trace 
(V).   V  -  ImAm/(l  +  Am). 

When  s  -  1,  i.e. ,  when  either  k  =  2  or  p  =  1 ,  the  four  criteria  lead 
to  identical  conclusions. 

The  robustness  of  MANOVA  is,  for  the  most  part,  an  unexplored 
topic.  Korin  (1972)  studied  the  behavior  of  R,  T,  and  W  under  the 
violation  of  the  homogeneity  of  covariance  matrices  assumption.  In 
his  study,  the  base  population  had  a  covariance  matrix  equal  to  I, 
the  identity  matrix.  The  other  populations  had  covariance  matrices 
in  the  form  of  d2I ,  where  d  =  JlTS ,  73.0,  or  710.0.  Korin  reported 
that  covariance  heterogeneity  in  small  samples,  even  when  the 
sample  sizes  are  all  equal,  produces  higher  Type  I  error  rates  than 
the  supposed  significance  level.  Among  these  three  test  criteria, 
R  has  the  highest  Type  I  error  rate.  He  did  not  include  V  test  in 
his  study. 

Olson  (1974)  compared  the  robustness  of  R,  T,  W,  and  V  to 
violations  of  the  homogeneity  of  covariance  matrices  assumption. 
In  his  study,  the  number  of  dimensions,  p,  was  2,  3,  6,  or  10,  the 
number  of  groups,  k,  was  2,  3,  6,  or  10,  and  the  sample  sizes,  n, 
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were  5,  10,  or  50.  Olson  referred  to  the  sample  drawn  from  a 
population  that  had  a  different  covariance  matrix  than  the  remaining 
populations  as  a  contaminated  group.  There  were  two  levels  of 
contamination,  namely,  low  concentration  of  contamination  and  high 
concentration  of  contamination.  In  the  low  concentration  condition, 
contamination  occurred  equally  in  all  dimensions  of  p-space.  Under 
high  concentration,  however,  contamination  occurred  in  only  one 
dimension  of  p-space.  The  degree  of  contamination,  d,  was  2,  3,  or 
6. 

Olson  (1974)  found  that  in  the  low  concentration  of 
contamination  condition,  all  of  the  four  criteria  became  liberal 
when  the  assumption  of  equal  covariance  matrices  was  violated.  The 
departure  from  the  nominal  a  can  be  serious  if  d  >  3.  The  order  of 
the  tests  in  terms  of  the  magnitude  of  the  departure  of  r  from  a  was 
typically  R  >  T  >  W  >  V.  The  size  of  the  departure  increased  as 
heteroscedasticity  increased.  With  increasing  values  of  p,  the  t's 
for  R,  T,  and  W  also  increased.  However,  t  for  V  did  not  respond 
in  any  clearcut  way  to  changes  in  p.  Increasing  sample  sizes 
decreased  t  for  R,  T,  and  W.  It  slightly  increased  r  for  V  when  k 
>  6.  However,  this  increased  t  for  V  was  still  smaller  than  those 
of  R,  T,  and  W. 

Under  the  high  concentration-of -contamination  condition,  Olson 
(1974)  found  that  the  departure  from  the  supposed  significant  level 
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was  mild,  even  though  one  dimension  was  severely  contaminated. 

Olson  (1974)  pointed  out  that,  above  all,  every  effort  should 
be  made  to  maintain  samples  of  equal  size,  or  at  least  to  prevent 
contaminated  groups  from  being  smaller  than  other  groups.  Olson 
concluded  that  the  V  test  is  the  most  robust  among  the  four  against 
heterogeneity  of  covariance  matrices,  although  its  Type  I  error  rate 
is  somewhat  greater  than  a. 

James'  and  Johansen's  Solutions 

James  (1954)  and  Johansen  (1980)  proposed  tests  to  deal  with 
the  multivariate  k-sample  Behrens-Fisher  problem.  James'  (1954) 
solution  is  an  extension  of  his  solution  to  the  two-sample 
multivariate  Behrens-Fisher  problem.  Johansbn  (1980),  on  the  other 
hand,  generalized  Welch's  (1951)  APDF  solution. 
James'  (1954)  Solution 

The  test  statistic  is 

k 

I      (x,  -  x)'W,(x,  -  x) 
i=l 

where , 


x  =  W1X,W,x, 
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and 


W,  =  (S/n,)"1 


W  =  X,w, 


The  critical  value  is 

2h(S2)  =  xf  (A  +  X2B) 
where 

r  =  p(k  -  1) 


A.1+  1    I        [tr(I  -W^lf 


i=l 


r(r  +  2) 


I 
1-1 


tr(I  -  vr1w,)2 


1  £   [tr(I  -  tr1w()]2 
1=1         ' 


James  (1954)  also  proposed  a  second  order  test.   It  was  presented 
as  formula  6.7  in  his  paper. 
Johansen's  (1980)  Solution 
The  test  statistic  is 


[X,(x,  -  x)'W,(x,  -  x)]/C 


where 
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C  -  p(k  -  1)  +  2A  -  6A/[p(p  -  1)  +  2 
and 


ttr(I  -  W%y   +  [tr(I  -  VT'W,) 

A  -   I     2^ 

i  =  l  ' 


The  critical  value  is  F  with  degrees  of  freedom  df,  and  df2, 
where 

df,  -  p(k  -  1) 

df2  =  p(k  -  l)[p(k  -  1)  +  2]/3A   . 

Ito  (1969)  studied  the  large  sample  behavior  of  James'  (1954) 
test.  He  used  total  sample  sizes  of  300  or  600.  For  k  =  3  and  5, 
he  found  that  James'  test  always  tends  to  result  in  a  slight 
overestimation  of  significance.  This  tendency  is  more  severe  for 
unequal  sample  sizes  than  equal  sample  sizes.  In  addition,  the 
discrepancy  from  nominal  a  increased  as  p  increased  from  1  to  4 . 
The  robustness  of  James  test  for  k  >  2  when  sample  sizes  are  small 
to  moderate  has  not  been  investigated. 

By  using  the  method  of  James  (1954)  to  derive  the  APDF  test 
of  Johanson  (1980),  Kaiser  (1983)  showed  that  equivalence  to  terms 
of  order  1/n  holds  between  the  solutions  of  Johansen  and  James. 
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Summary 
Based  on  the  preceding  review,  one  can  conclude  that  t,  F,  T  , 
and  MANOVA  criteria  are  not  robust  to  violation  of  the  assumption 
of  equal  variance  (covariance  matrices  in  the  multivariate  case) 
when  sample  sizes  are  not  equal.  Tests  proposed  to  deal  with  the 
Behrens- Fisher  problem,  on  the  other  hand,  are  more  robust  than  the 
usual  tests.  These  tests  follow  one  of  two  approaches.  One  is  to 
develop  the  critical  value  of  the  test  statistic  through  a  series 
expansion  (series  solution).  The  other  approach  is  to  approximate 
the  distribution  of  a  test  statistic  through  approximate  degrees  of 
freedom  (APDF  solution) .  Results  of  these  two  approaches  agree  to 
order  1/n  .  The  solutions  to  the  multivariate  problem  are 
generalizations  of  the  univariate  solution. 

In  general,  the  tests  designed  to  deal  with  the  Behrens-Fisher 
problem  are  liberal,  i.e.,  they  tend  to  have  higher  Type  I  error 
rates  than  the  nominal  a.  However,  this  liberal  tendency  is  mild 
except  when  the  sample-size  ratios  are  extreme.  Therefore,  these 
tests  generally  perform  much  better  than  the  usual  tests  in 
controlling  the  Type  I  error  rate  when  the  assumption  of  homogeneity 
of  variance  (covariance  matrices  in  the  multivariate  case)  is 
violated . 

The  preceding  studies  indicate  that  APDF  solutions  are  more 
effective  in  controlling  type  I  error  rate  than  the  series  solutions 
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of  the  first  order.  However,  in  the  univariate  case  when  a  second 
order  series  solution  is  used,  its  type  I  error  rate  is  closer  to 
a  than  that  of  the  APDF  solution. 


CHAPTER  3 
METHODOLOGY 


The  design  of  the  study  and  the  simulation  procedure  employed 
to  carry  out  the  study  are  described  in  this  chapter.  Four  factors 
are  the  most  important  in  designing  a  multivariate  experiment: 
number  of  populations  sampled,  sample  sizes,  number  of  variables, 
and  the  significance  level.  The  design  of  the  present  study  was 
based  on  these  four  factors.  In  addition,  the  degree  and  form  of 
heterogeneity  was,  of  course,  studied. 

Design 

The  design  of  the  study  was  based  on  the  consideration  that 
experimental  conditions  included  in  the  simulation  should  reflect 
the  reality  of  multivariate  experiments  in  educational  research. 

Number  of  dimensions  (p)  .  Data  were  generated  to  simulate 
experiments  in  which  there  are  p  =  3  or  p  =  6  response  variates. 
It  seems  likely  that  p  >  6  occurs  seldomly  in  educational  research, 
and  therefore  p  =  6  was  the  upper  limit  chosen  for  the  study. 

Number  of  populations  sampled  (k)  .  All  of  the  simulated 
experiments  had  a  simple  one-way  design  with  k  =  3.  Algina  and  Tang 
(1988)  investigated  the  behavior  of  James'  (1954)  and  Yao's  (1965) 
tests  for  k  =  2.   Therefore,  the  present  study  will  concentrate  on 
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investigating  the  behavior  of  James'  and  Johansen's  (1980)  tests 
when  k  >  2.  In  the  univariate  literature  (Dijkstra  &.  Werter,  1981), 
it  appears  that  the  advantage  of  James'  second  order  test  is  in 
experiments  with  many  groups.  However,  experiments,  especially 
multivariate  experiments,  with  a  large  number  of  groups  seem  to  be 
rare  in  educational  research.  Therefore,  k  was  kept  at  three.  The 
simplification  of  this  factor  allowed  more  levels  of  another  factor, 
the  form  of  covariance  matrix  heteroscedasticity ,  to  be  investigated 
more  thoroughly. 

Sample-size  ratios  (n1:ru:n3).   Both  equal  and  unequal  sample 
sizes  were  used  in  the  study.   When  the  sample  sizes  were  unequal, 
the  sample-size  ratios  were  small  to  moderately  large.   The  basic 
ratios  of  n,:^:^  used  in  the  simulation  are  presented  in  Table  1. 
Table  1 
Sample-Size  Ratios  (n^ru,:^) 


"2  :  n3 


1:1:1 
1  :  1  :  1.3 
1:1:2 

1  :  1.3  :  1.3 
1:2:2 


32 
In  some  cases  these  basic  ratios  could  not  be  maintained  exactly 
because  of  the  restriction  of  a  fixed  total  sample  size  (N) .  The 
departure  from  these  basic  ratios  was  made  as  small  as  possible. 

Ratio  of  total  sample  size  (N)  to  number  of  variables.  The 
ratio  N  to  p  was  10,  15,  or  20.  Total  sample  sizes  of  lOp,  15p,  or 
20p  was  assigned  to  k  groups  based  on  the  ratio  illustrated  in  Table 
1.  The  sample  sizes  used  in  the  present  study  are  displayed  in 
Tables  2  and  3.  Algina  and  Tang  (1988)  studied  the  case  of  k  -  2 
samples  and  included  N:p  =  6,  10,  and  20.  They  reported  that  N:p 
should  be  at  least  10  to  be  reasonably  sure  that  Yao's  test,  which 
was  generally  more  robust  than  James'  test,  would  be  robust.  With 
k  >  2,  it  seems  likely  that  N:p  will  need  to  be  at  least  10  for 
robustness  to  obtain.  Therefore,  in  the  present  study,  the  upper 
limit  of  20  was  chosen  to  represent  moderately  large  experiments: 
a  total  sample  size  of  60  when  p  =  3  and  a  total  sample  size  of  120 
when  p  =  6 . 

Heterogeneity  of  covariance  matrices.  Each  population  with 
a  covariance  matrix  equal  to  a  p  x  p  identity  matrix  (I)  will  be 
called  an  "uncontaminated"  population.  Each  population  with  a  p  x 
p  diagonal  covariance  matrix  D  with  at  least  one  diagonal  element 
not  equal  to  one  will  be  called  a  "contaminated"  population.  Three 
forms  of  D  were  used  in  this  study.   They  are  shown  in  Table  4. 
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Table  2 

Sample  Sizes  When  p  =  3 

N/p    n,     n2    n3 


10 

9 

9 

12 

10 

7 

7 

16 

10 

8 

11 

11 

10 

6 

12 

12 

15 

13 

13 

19 

15 

11 

11 

23 

15 

13 

16 

16 

15 

9 

18 

18 

20 

18 

18 

24 

20 

15 

15 

30 

20 

16 

22 

22 

20 

12 

24 

24 

Two  levels  of  d,  d  =  JTTb  and  3.0,  were  employed  to  simulate 
heterogeneity  of  covariance  matrices.  In  Algina  and  Tang's  (1988) 
study,  d  was  chosen  to  be  1.5,  2.0,  2.5,  or  3.0.   In  Olson's  (1974) 
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Table  3 

Sample  Sizes  When  p  =  6 


N/p 


n2 


10 

18 

18 

24 

10 

15 

15 

30 

10 

16 

22 

22 

10 

12 

24 

24 

15 

27 

27 

36 

15 

22 

22 

46 

15 

26 

32 

32 

15 

18 

36 

36 

20 

36 

36 

4  8 

20 

30 

30 

60 

20 

32 

44 

44 

20 

24 

48 

48 

study,  d  was  chosen  to  be  2.0,  3.0,  or  6.0.  They  found  that  the 
smaller  the  d  value,  the  more  robust  the  test.  In  the  present 
study,  d  =  J\ .  5  was  used  to  simulate  a  small  degree  of 
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Table  4 

Forms  of  Covariance  Matrix  D 


Form 


1  diag(d2,  d2,  d2)     diag(d2,  d2,  d2,  d2,  d2,  d2) 

2  diag(l,  d2,  d2)     diag(l,  1,  1,  d2,  d2,  d2) 

3  diag(l/d2,  d2,  d2)   diag(l/d2,  1/d2,  1/d2,  d2,  d2,  d2) 


heteroscedasticity ,  which  seems  more  likely  to  be  common  in 
educational  experiment  than  extreme  heteroscedasticity,  such  as  d 
=  6.0,  In  the  present  study,  d  =  3.0  was  selected  to  investigate 
a  larger  degree  of  heteroscedasticity  and  to  permit  comparison  of 
the  results  of  the  present  study  with  those  of  Algina  and  Tang 
(1988) ,  and  Olson  (1974)  . 

Two  relationships  between  sample  size  and  covariance  matrices 
were  employed.  In  the  positive  relationship,  the  larger  samples 
were  associated  with  D.  In  the  negative  relationship,  the  smaller 
samples  were  associated  with  D.  These  two  relationships  are 
summarized  in  Table  5.  For  equal  sample  sizes,  the  two  nonredundant 
heteroscedasticity  patterns,  (I,  I,  D)  or  (I,  D,  D) ,  were  employed. 

Significance  levels.  Three  levels  of  significance  were 
employed:  .10,  .05,  and  .01. 
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Table  5 

Combination  of  Sample-Size  Ratios  and  Contamination 


Sample-Size  Ratios  Relationship 


n,  :  rv,  :  n3 


Positive     Negative 


1.0  :  1.0  :  1.3 

1.0  :  1.0  :  2.0 

1.0  :  1.3  :  1.3 

1.0  :  2.0  :  2.0 


I    I    D 

D   D    I 

I    I    D 

D   D    I 

I    D   D 

D    I    I 

I    D   D 

D    I    I 

Invariance  Property  of  the  Test  Statistic 
The  form  of  heteroscedasticity  investigated  in  this  study  is 
that  the  populations  in  an  experiment  can  be  divided  into  two 
subsets;  each  subset  has  a  common  variance-covariance  matrix,  with 
the  matrices  varying  across  subsets.  In  the  simulated  experiments 
the  variance-covariance  matrix  for  one  subset  was  an  identity 
matrix,  whereas  the  matrix  for  the  second  subset  was  a  diagonal 
matrix.  These  matrix  forms  entail  no  loss  of  generality  beyond  that 
due  to  the  limited  form  of  heteroscedasticity  investigated  in  this 
study.   The  reasons  there  is  no  additional  loss  of  generality  are 
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twofold.  First,  by  a  well-known  theorem  (Anderson,  1958),  for  any 
pair  of  positive  definite  matrices  there  exists  a  matrix  C  which  can 
transform  the  matrices  into  an  identity  matrix  and  a  diagonal  matrix 
simultaneously.  Second,  as  shown  below,  the  test  statistics  are 
invariant  under  transformations  y,  =  Cx(,  i  -  1,  .  .  .  ,k,  where  x,  is 
the  observed  mean  vector,  and  C  is  a  p  x  p  nonsingular  matrix.  In 
this  section,  the  invariance  property  of  the  test  statistic  and  the 
critical  values  are  proved. 

The  estimated  variance -covariance  matrix  of  x,  is  A(  =  S/n . 
That  of  y  is  CA^'  .  The  test  statistic  of  both  James'  and  Johansen's 
procedure  is 

t'V1t  -  X|(x"i  -  x)'W,(X|  -  x)  ,  i  -  1 k 

where 

v,  =  V , 

and 

W  =  £,W, ,  i  =  1 k ; 

f  =   (f,  t'j  ...  t'K1), 

t,  -  x.  -  x„,  i  -  1,  .  .  .  ,k  -  1. 


and 
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The  covariance  matrix  of  t  is 

A,  +  A.      A, 


Cov  (t)  =  V 


K  *2  +  \ 


Ak  A^  +  Ak 

Let  t*  -  Ct  and  M  =  I  ®  C,   where  ®  denotes  the  Kronecker  product 
Then  t"  -  Mt ,  and 
Cov  (t*)  =  Cov  (Mt) 

CCA,  +  Ak)C  CAkC  CAkC 

CCA;,  +  Ak)C 


CAkC 
MVM'  =  V'. 


CAkC 
CAkC  C(AM  +  Ak)C 


Now, 
t*'V1t* 


(Mt)'  (MVM'  )"1(Mt) 
-  t'M'M"TV1M"1Mt, 
=  t'V1t. 


Therefore  the  test  statistic  is  invariant  under  nonsingular 
transformation . 
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For  both  James'  first  order  and  Johansen's  critical  values, 
the  quantities  that  are  relevant  to  the  invariance  property  are 

G  =  tr(I  -  r1W,)r,  r  -  1,  2,  3,  ... 

The  critical  value  of  James'  second  order  procedure  involves 
G  and  traces  of  cyclical  products  such  as 

Let  A,  j  =  1,  ...,pbe  the  eigenvalues  of  I  -  W  W( . 
Therefore  \T.,    j  -  1,  ...,  p  are  the  eigenvalues  of  (I  -  V1W,)r. 
We  need  only  to  show  that  the  eigenvalues  of  I  -  W^W,  are 
invariant,  since  tr(I  -  r1W,)r  =  ][,A[,  J  =  1.---P- 

The  As  are  the  solutions  to  the  equation 

1 1  -  vr1w,  -  XI |  =0,  or 
\V\    -    (1  -  A)l|  -  0 

For  the  transformed  variables  we  have 

|CTH"1C-1CW,C'  -  (1  -  A*)  1 1  =0,  or 
|C-TW1W,C  -  (1  -  A")  1 1  =0. 

If  we  premultiply  by  C  and  postmultiply  by  C,  the  eigenvalues 
remain  unchanged.   Carrying  out  these  multiplications  gives 
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|W1W  -  (1  -  A*) 1 1  =0,  which  proves  A  =  A* .   Therefore 
G  =  tr(I  -  W"1W()r  is  invariant. 

By  the  property  that  tr(XY)  -  tr(YX) ,  it  is  obvious  that 

tr(E*)  -  trCCVW^WjW^jW'WjC') 

=  tr(E) 

and  that  the  trace  of  other  cyclical  products  of  W',W|  and  W1W|  will 
also  be  invariant. 

By  the  preceding  results,  using  only  diagonal  matrices  to 
simulate  experiments  in  which  there  are  only  two  sets  of  covariance 
matrices  is  justified.  However,  when  there  are  more  than  two  sets 
of  different  matrices,  the  matrices  cannot  always  be  simultaneously 
diagonalized  by  a  matrix  C. 

Simulation  Procedure 

For  each  condition,  data  were  generated  and  the  performance 
of  the  Pillai-Bartlett  (V)  test,  James'  first  order  test,  James' 
second  order  test,  and  Johansen's  tests  were  evaluated  using  the 
generated  data. 

Generating  mean  vectors.  In  order  to  generate  a  mean  vector, 
a  p  x  1  vector  of  uncorrelated  standard  normal  pseudorandom 
variables  was  generated  by  using  the  RANNOR  function  in  SAS  (1985). 
Multiplying  the  vector  by  l/Tn^,  a  sample  mean  vector  from  an 
uncontaminated  population  was  obtained.   Multiplying  the  vector  by 
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(l/JrT)D"' ,    a  sample  mean  vector  from  a  contaminated  population  was 
obtained. 

Generating  covariance  matrices.  Let  T  be  a  p  x  p  lower 
triangular  matrix.  Kshirsagar  (1959)  showed  that  if  T,,  ~  N(0,1), 
for  i  *  j ,  and  T2  -  x2nr  wnere  n  is  tne  number  of  observations  of 
a  sample,  then  TT'  -  W(n  -  1,  Ip)  ,  a  Wishart  distribution  with 
n  -  1  degrees  of  freedom.  Thus  S  =  (n  -  1)"1TT'  is  a  covariance 
matrix  for  a  sample  of  size  n  from  a  population  with  covariance 
matrix  I.  That  is,  S  is  a  sample  covariance  matrix  from  an 
uncontaminated  population.  Pre  and  post  multiplying  S  by  D'  ,  a 
covariance  matrix  from  a  contaminated  population  is  obtained.  By 
using  the  RANNOR  function  described  above,  the  T„s  are  generated; 
by  using  the  RAN  GAM  function,  the  T2,s  are  generated. 

Evaluating  the  performance  of  test  statistics.  For  each 
replication,  the  data  generated  were  analyzed  by  the  Pillai -Bartlett 
trace  (V),  James'  first  order  and  second  order  test,  and  Johansen's 
tests.  The  test  statistics  and  the  critical  values  were  calculated 
using  PROC  MATRIX  in  SAS .  The  proportion  of  2000  replications  that 
yield  significant  results  at  a  -  .10,  .05,  and  .01  was  recorded. 

Summary 
Combining  one  level  of  k  (k  =  3),  two  levels  of  p  (p  -  3  or 
6),  four  levels  of  sample-size  ratios  (see  Table  1),  three  levels 
of  N/p  (10,  15,  20),  two  levels  of  d  (d  =  JTTb   or  3.0),  three  levels 
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of  D,  and  two  levels  of  forms  of  contamination  (positive  or 
negative),  288  experimental  conditions  are  obtained  with  unequal 
sample  sizes.  In  addition  there  were  72  conditions  with  equal 
sample  sizes.  The  Pillai -Bartlett  trace  (V),  James'  first  order  and 
second  order  tests,  and  Johansen's  test  were  applied  in  these 
experimental  conditions.  A  general  picture  of  the  behavior  of  these 
tests  was  obtained  by  these  360  simulated  experiments. 


CHAPTER  4 
RESULTS  AND  DISCUSSIONS 


In  this   chapter,   the  results   for  nominal  a     -      .05  are 

presented.   In  general,  the  major  patterns  of  results  are  similar 

for  a   =    .01,  .05,  and  .10.   Consequently,  the  results  for  a  =  . 01 

and  a  -  .10  are  tabled  in  Appendices  A,  B,  and  C.   The  chapter  is 

divided  into  two  major  sections.   In  the  first  section  the  results 

are  presented  for  conditions  with  equal  sample  sizes.   The  results 

for  conditions  with  unequal  sample  sizes  are  reported  in  the  second 

section  of  the  chapter. 

The  Performance  of  the  Four  Multivariate  Tests 
When  n1  =  ru  =  n3 

The  Performance  of  the  Four  Tests  When  d  =  Jl . 5 

Estimated  rs  are  reported  in  Tables  6,   7,  and  8  for  the 

Pillai-Bartlett  trace,  Johansen's  test,  James'  first  order  test,  and 

James'   second  order  test;   these  results  are  for  a  =   .05  in 

conditions  in  which  n,  =  ru,  =  n3  and  d  =  Jl.5 .       Thirty-six  ts  are 

reported  for  each  test.   For  the  Pillai-Bartlett  trace,  one  of  the 


rs  falls  outside  the  criterion  interval  a  ±  2ja( l-a)/2000 .  For 
Johansen's  test,  five  rs  fall  outside  the  criterion  interval.  The 
largest  r  was  .065;  four  of  the  five  occur  when  n,  -  10,  i  -  1,  2, 
3.  The  number  of  rs  outside  the  interval  is  21  and  16  for  James' 
first  order  test  and  second  order  test  respectively.   For  James' 
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uu 


Table  6 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
n1  _^_  ru  =  n3.  D  =  d2I .  d  =  7175.  and  a  =  .05 


N/p      Pillai-     Johansen's     James'      James' 
Bartlett     test       1st  order   2nd  order 
trace  test       test 


I:I:D 

3 

10 

.0450 

.0460 

.0605 

.0330 

3 

15 

.0485 

.0530 

.0615 

.0455 

3 

20 

.0520 

.0560 

.0615 

.0505 

6 

10 

.0445 

.0520 

.0635 

.0330 

6 

15 

.0480 

.0510 

.0550 

.0365 

6 

20 

.0475 

.0460 

.0495 

.0375 

I:D:D 

3 

10 

.0430 

.0540 

.0665 

.0385 

3 

15 

.0495 

.0565 

.0635 

.0500 

3 

20 

.0485 

.0535 

.0570 

.0480 

6 

10 

.0455 

.0560 

.0695 

.0255 

6 

15 

.0450 

.0475 

.0530 

.0375 

6 

20 

.0555 

.0605 

.0655 

.0500 

Note. 

The 

underlined  ts 

are  outside 

the  interval 

a   ±  27a(l-Q)/2000. 
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Table  7 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
n1_=_ga_^_%J  D  -  diae  (d2  d2  1)  or  D  -  diag  (d2  d2  d2  1  1  1)  . 
d  =  JlTb.    and  a  =  .05 


N/p     Pillai-    Johansen's     James'      James' 

Bartlett     test       1st  order    2nd  order 
trace  test        test 


I:I:D 

3 

10 

.0455 

.0625 

.0745 

.0430 

3 

15 

.0465 

.0540 

.0585 

.0490 

3 

20 

.0525 

.0555 

.0620 

.0500 

6 

10 

.0570 

.0650 

.0855 

.0355 

6 

15 

.0545 

.0590 

.0665 

.0420 

6 

20 

.0570 

.0540 

.0580 

.0455 

I:D:D 


3 

10 

.0480 

.0610 

.0755 

.0410 

3 

15 

.0505 

.0555 

.0660 

.0455 

3 

20 

.0465 

.0480 

.0540 

.0460 

6 

10 

.0435 

.0565 

.0715 

.0275 

6 

15 

.0435 

.0475 

.0525 

.0350 

6 

20 

.0495 

.0525 

.0570 

.0390 

Note  . 

The 

unde 

rlined  fs 

are  outside 

the 

interval 

q  ±  2ya(l-a)/2000. 
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Table  8 

Actual  Tvpe  I  Error  Rates  for  Four  Multivariate  Tests: 

B,   -  il.  -  n3.    D  -   diac    (d2   d2   1/d2)    or 

D  -   diag    (d2  d2  d2   1/d2   1/d2   1/d2).    d  =  ./I75.    and  a  -    .05 


I:I:D 


I:D:D 


N/p      Pillai-      Johansen's     James'       James' 
Bartlett      test      1st  order   2nd  order 
trace  test        test 


1 

10 
15 

.0470 

.0505 

.0555 

.0515 

.0730 

.0380 

3 

.0565 

.0440 

3 

20 

.0385 

.0435 

.0475 

.0400 

6 

10 

.0465 

.0515 

.0675 

.0265 

6 

15 

.0450 

.0555 

.0620 

.0400 

6 

20 

.0570 

.0570 

.0620 

.0515 

3 

10 

.0515 

.0535 

.0660 

.0420 

3 

15 

.0515 

.0510 

.0595 

.0455 

3 

20 

.0405 

.0500 

.0555 

.0455 

6 

10 

.0505 

.0625 

.0790 

.0405 

6 

15 

.0485 

.0475 

.0535 

.0365 

6 

20 

.0565 

.0535 

.0580 

.0440 

Note .   The  underlined  fs  are  outside  the  interval 
a   ±  2ya(l-o)/2000. 
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first  order  test,  the  21  rs  outside  the  interval  are  larger  than  a 
-  .05.  The  fs  for  the  first  order  test  tend  to  be  larger  than  those 
for  Johansen's  test.  For  James'  second  order  test,  the  16  fs 
outside  the  interval  are  less  than  a  =    .05. 

James'  second  order  test  exhibited  some  tendency  to  perform 
better  with  p  -  3  than  with  p  =  6.  However,  in  general  p  did  not 
appear  to  have  a  strong  effect  for  any  of  the  tests.  Neither,  did 
the  pattern  of  contamination  (I:I:D  vs  I:D:D).  As  N/p  increased, 
there  was  some  tendency  for  the  performance  of  James'  and  Johansen's 
tests  to  improve.  The  second  order  test  appears  to  have  f  nearer 
a  for  D  *  d2I  than  for  D  =  d2I .  The  first  order  test  and  Johansen's 
test  did  not  appear  to  be  affected  by  the  form  of  D. 
The  Performance  of  the  Four  Tests  When  d  -  3 

In  Tables  9,  10,  and  11,  36  rs  are  reported  for  each  test 
under  the  condition  in  which  n,  -  tv,  -  n3  and  d  =  3.  For  the  Pillai- 
Bartlett  trace  29  fall  outside  the  criterion  interval.  For 
Johansen's  test,  12  fall  outside  the  interval.  All  of  these  are 
larger  than  .05.  Eight  of  rs  outside  the  interval  occur  in 
conditions  in  which  n,  =  10.  For  James'  first  order  test,  22  fall 
outside  the  interval;  all  are  larger  than  .05.  For  James'  second 
order  test,  17  fall  outside  the  interval;  all  are  smaller  than  .05. 

In  general,  the  effects  of  the  factors  on  James'  and 
Johansen's  tests  are  similar  to  their  effects  for  d  =  JlTS . 
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Table  9 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
n1  =  ru  =  n3.  D  =  d2I ,  d  =  3,  and  a  =  .05 


N/p     Pillai-    Johansen's     James'       James' 

Bartlett     test      1st  order    2nd  order 
trace  test        test 


I:I:D 

3 

10 

.0920 

.0665 

.0830 

.0430 

3 

15 

.0855 

.0530 

.0590 

.0425 

3 

20 

.0890 

.0555 

.0595 

.0505 

6 

10 

.0860 

.0690 

.0790 

.0395 

6 

15 

.0905 

.0550 

.0635 

.0395 

6 

20 

.0795 

.0435 

.0505 

.0320 

I:D:D 

3 

10 

.0505 

.0585 

.0800 

.0315 

3 

15 

.0615 

.0575 

.0665 

.0425 

3 

20 

.0625 

.0520 

.0580 

.0435 

6 

10 

.0635 

.0770 

.0875 

.0325 

6 

15 

.0655 

.0610 

.0680 

.0340 

6 

20 

.0690 

.0605 

.0665 

.0475 

Note  . 

The 

underlined  fs 

are  outside 

the  interval 

a  ±  2yQ(l-Q)/2000. 
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Table  10 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 

n1  =  ru  =  n3.  D  =  diag  (d2  d2  1)  or 

D  -  diae  (d2  d2  d2  1  1  1  ) .  d  =  3.  and  a   -    .05 


p     N/p    Pillai-    Johansen's     James'       James' 

Bartlett     test      1st  order   2nd  order 
trace  test        test 


I:I:D 


3 

10 

.0715 

.0590 

.0775 

.0430 

3 

15 

.0685 

.0405 

.0470 

.0350 

3 

20 

.0640 

.0450 

.0485 

.0390 

6 

10 

.0725 

.0640 

.0800 

.0315 

6 

15 

.0680 

.0475 

.0560 

.0365 

6 

20 

.0765 

.0515 

.0545 

.0410 

I:D:D 


3 

10 

.0535 

.0590 

.0750 

.0385 

3 

15 

.0595 

.0610 

.0665 

.0510 

3 

20 

.0590 

.0490 

.0540 

.0430 

6 

10 

.0480 

.0580 

.0720 

.0280 

6 

15 

.0460 

.0535 

.0560 

.0375 

6 

20 

.0560 

.0470 

.0505 

.0395 

Note  . 

The 

underlined  fs 

are  outside 

the 

interval 

a   ±  2yQ(l-Q)/2000. 


50 


Table  11 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 

n,    -   ru   -   n3.       D   -   diag    ( d2   d2   1/d2) 

D  -  diag    (d2  d2   d2   1/d2   1/d2   1/d2)  .    d  -   3 .    and  a   -    .05 


N/p     Pillai-      Johansen's     James'       James' 
Bartlett      test      1st  order   2nd  order 
trace  test        test 


I:I:D 


3 

10 

.0780 

.0665 

.0810 

.0460 

3 

15 

.0735 

.0520 

.0605 

.0425 

3 

20 

.0740 

.0475 

.0515 

.0430 

6 

10 

.0735 

.0675 

.0800 

.0370 

6 

15 

.0760 

.0665 

.0755 

.0480 

6 

20 

.0775 

.0505 

.0545 

.0425 

I:D:D 


3 

10 

.0705 

.0700 

.0845 

.0365 

3 

15 

.0745 

.0580 

.0655 

.0400 

3 

20 

.0710 

.0505 

.0565 

.0410 

6 

10 

.0665 

.0625 

.0785 

.0440 

6 

15 

.0735 

.0525 

.0610 

.0435 

6 

20 

.0685 

.0540 

.0600 

.0485 

Note  . 

The 

und 

srlined  fs 

are  outside 

the 

interval 

a  ±  2yQ(l-a)/2000. 
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However,  the  effect  of  p  on  the  second  order  test  appears  to  be 
stronger  when  d  =  3  than  when  d  =  71 . 5 . 
Summary 

The  Pillai-Bartlett  test  worked  well  when  the  degree  of 
heterogeneity  was  small,  d  =  Jl .  5 ,  but  had  excessive  type  I  error 
rates  when  the  degree  of  heterogeneity  was  large,  d  =  3.0. 
Johansen's  test  worked  reasonably  well,  particularly  with  N/p  >  15. 
Consequently,  it  probably  should  be  use  in  place  of  the  usual 
multivariate  criteria.  James'  first  order  test  is  not  competitive; 
it  had  more  fs  than  Johansen's  test  had  outside  the  interval  and 
tended  to  have  t  larger  than  that  for  Johansen's  test.  James' 
second  order  test  had  more  fs  than  Johansen's  test  had  outside  the 
criterion  interval.  This  seems  to  favor  Johansen's  test.  However, 
some  researchers  may  favor  the  second  order  test  because  this  test 
tended  to  have  f  smaller  than  a,  whereas  Johansen's  test  tended  to 
have  t    larger  than  a. 

The  Performance  of  the  Four  Multivariate  Tests 
When  Sample  Sizes  Are  Unequal 

Estimated  Type  I  error  rates  are  reported  for  conditions  in 
which  the  sample  sizes  are  unequal.  The  section  is  divided  into 
three  subsections  defined  by  N/p  (10,  15,  or  20).  In  each 
subsection,  96  fs  are  reported  for  each  of  the  four  tests:  Pillai- 
Bartlett,  Johansen,  James'  first  order,  and  James'  second  order. 
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For  each  subsection,  there  are  eight  tables,  one  for  each 
combination  of  p  (3  or  6),  direction  of  relationship  between  the 
sample  sizes  and  covariance  matrices  (positive  or  negative),  and  d 
(TO  or  3.0). 
The  Performance  of  the  Four  Tests  When  N/p  =  10 

Forty  eight  estimated  rs  for  each  of  the  four  tests  for  the 
conditions  in  which  N/p  =  10  and  d  -  JTTb  are  reported  in  Tables  12 
through  15.  The  number  of  fs  outside  the  interval  is  20  for  the 
Pillai-Bartlett  Trace,  30  for  the  second  order  test,  37  for 
Johansen's  test,  and  48  for  the  first  order  test.  Estimated  rs  for 
conditions  in  which  d  =  3.0  are  reported  in  Tables  16  through  19. 
The  number  of  fs  outside  the  criterion  interval  is  44,  31,  37,  and 
48  for  the  Pillai-Bartlett  trace,  James'  second  order  test, 
Johansen's  test,  and  James'  first  order  test  respectively.  For  the 
Pillai-Bartlett  trace,  when  d  changes  from  JT.  5  to  3.0,  the  increase 
in  the  number  of  rs  outside  the  criterion  interval  is  120%.  The 
decline  in  performance  is  most  notable  for  the  negative  condition 
in  which  only  five  are  outside  the  criterion  interval  for  d  =  Jl . 5 
but  all  24  fs  are  outside  the  criterion  interval  for  d  =  3.0.  For 
the  other  tests  the  number  of  fs  outside  the  criterion  interval  does 
not  change  dramatically  when  d  increases  from  JYTb  to  3.  Based  on 
the  number  of  fs  outside  the  criterion  interval  it  appears  that 
there  was  only  a  slight  decline  in  the  performance  of  Johansen's 
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Table  12 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  10.  p  =  3.  d  =  71.5.  and  a  -  .05 


^2  n3 


Pillai-     Johansen's     James'       James' 
Bartlett     test       1st  order   2nd  order 
trace  test        test 


.0625  .0375 

.0930  .0455 

.0620  .0380 

.0955  .0540 


D  - 

d2I 

9 

9 

12 

.0375 

.0520 

7 

7 

16 

.0240 

.0735 

8 

11 

11 

.0380 

.0530 

6 

12 

12 

.0380 

.0765 

D  = 

diag 

(d2 

d2  1) 

9 

9 

12 

.0345 

.0515 

7 

7 

16 

.0285 

.0675 

8 

11 

11 

.0370 

.0520 

6 

12 

12 

.0430 

.0775 

D  = 

diag 

(d2 

d2  1/d2) 

9 

9 

12 

.0410 

.0580 

7 

7 

16 

.0325 

.0705 

8 

11 

11 

.0435 

.0610 

6 

12 

12 

.0455 

.0765 

.0640        .0350 


.0920       .0370 


.0735       .0365 
.0990       .0515 

.0725       .0375 


0965        .0445 


.0780        .0375 
.0970        .0485 


Note .  When  n,  =  r^  *  n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *  n2  -  n3,  the  form  of  contamination  is  I:D:D.  The  underlined  fs 
are  outside  the  interval  a   ±   2ja(l-a) /2000 . 
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Table  13 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 


Negative 

Cor 

dition , 

N/P  - 

10.  p 

-  3.  d  ■ 

>  jr. 

5 ,  and  a 

-  .05 

ni 

*2 

n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  -  d2I 

9 

9 

12 

.0540 

.0555 

.0720 

.0395 

7 

7 

16 

.0725 

.0845 

.1115 

.0540 

8 

11 

11 

.0535 

.0630 

.0820 

.0435 

6 

12 

12 

.0580 

.0810 

.1035 

.0495 

D  =  diag 

(d2 

d2  1) 

9 

9 

12 

.0475 

.0575 

.0785 

.0385 

7 

7 

16 

.0580 

.0725 

.0980 

.0490 

8 

11 

11 

.0540 

.0685 

.0880 

.0460 

6 

12 

12 

.0675 

.0855 

.1100 

.0590 

D  =  diag 

(d2 

d2  1/d2) 

9 

9 

12 

.0435 

.0630 

.0845 

.0425 

7 

7 

16 

.0530 

.0695 

.0875 

.0420 

8 

11 

11 

.0525 

.0610 

.0745 

.0415 

6 

12 

12 

.0580 

.0825 

.1030 

.0490 

Note .  When  n,  =  r^  *  n3 ,  the  form  of  contamination  is  D:D:I;  when 
n,  /  rij  =  n3,  the  form  of  contamination  is  D:I:I.  The  underlined  rs 
are  outside  the  interval  a  ±  2ja(l— a)/2000 . 
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Table  14 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  10,  p  =  6,  d  =  ./l . 5 ,  and  a  =  .05 


ru     n3        Pillai-  Johansen's  James'  James' 

Bartlett     test       1st  order   2nd  order 
trace  test        test 


0695       .0280 


D  - 

-  d2I 

18 

18 

24 

.0335 

.0530 

15 

15 

30 

.0255 

.0645 

16 

22 

22 

.0335 

.0560 

12 

24 

24 

.0320 

.0635 

D  = 

=  diag 

(d2 

d2  d2  1  1  1) 

18 

18 

24 

.0410 

.0615 

15 

15 

30 

.0380 

.0835 

16 

22 

22 

.0375 

.0650 

12 

24 

24 

.0400 

.0685 

.0820  .0375 

.0685  .0265 

.0820  .0295 

.0755  .0335 


.1000 

.0335 

.0770 

.0330 

^0900 

.0330 

D  =   diag(d2   d2   d2   1/d2   1/d2   1/d2) 

18   18   24     .0470       .0690  .0870  .0320 

15  15   30     .0445       .0715  .0920  .0280 

16  22   22     .0510       .0630  .0730  .0330 
12   24   24     .0540        .0805  .0995  .0405 


Note .  When  n,  =  r^  ^  n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  ¥■  r^  -=  n3,  the  form  of  contamination  is  I:D:D.  The  underlined  rs 
are  outside  the  interval  a  ±  27a( l-a)/2000 . 


56 


Table  15 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =  10.  p  -  6.  d  =  71.5.  and  a   =    .05 


r^      n3        Pillai-  Johansen's  James'  James' 

Bartlett     test       1st  order   2nd  order 
trace  test        test 


D  - 

■  d2I 

18 

18 

24 

.0530 

.0565 

15 

15 

30 

.0725 

.0675 

16 

22 

22 

.0595 

.0720 

12 

24 

24 

.0825 

.0940 

D  ■ 

•  diag 

(d2 

d2  d2 

1  1  1) 

18 

18 

24 

.0570 

.0595 

15 

15 

30 

.0560 

.0720 

16 

22 

22 

.0585 

.0700 

12 

24 

24 

.0670 

.0865 

D  ■ 

-  diag 

(d2 

d2  d2 

1/d2  1/d2 

1/d2) 

18 

18 

24 

.0560 

.0670 

15 

15 

30 

.0500 

.0680 

16 

22 

22 

.0475 

.0675 

12 

24 

24 

.0540 

.0790 

.0670       .0290 


.0870        .0315 
.0855       .0355 


.1190       .0425 


0735       .0290 


.0895       .0305 


.0840       .0365 


1070       .0425 

0825       .0360 
.0865       .0310 


.0820       .0335 


.0995       .0305 


Note.  When  n,  -  n,  *  n3,  the  form  of  contamination  is  D:D:I;  when 
n,  *  r^  =  n3,  the  form  of  contamination  is  D:I:I.  The  underlined  fs 
are  outside  the  interval  a  ±  2ya(l-a)/2000 . 
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Table  16 


Positive 

Con 

dition. 

N/p  - 

10.  p 

=  3.  d  - 

-  3 ,  and  a  = 

.05 

ni 

n„ 

n3 

Pi 

llai- 

J  oh 

arisen'  s 

James ' 

James ' 

Bartlett 

test 

1st  order 

2nd 

order 

trace 

test 

test 

D  -  d2I 

9 

9 

12 

.0365 

.0525 

.0645 

.0360 

7 

7 

16 

.0065 

.0565 

.0775 

.0375 

8 

11 

11 

.0375 

.0675 

.0815 

.0460 

6 

12 

12 

.0200 

.0605 

.0750 

.0455 

D  =  diag 

(d2 

d2  1) 

9 

9 

12 

.0425 

.0580 

.0740 

.0365 

7 

7 

16 

.0125 

.0585 

.0785 

.0390 

8 

11 

11 

.0355 

.0660 

.0810 

.0450 

6 

12 

12 

.0280 

.0520 

.0695 

.0360 

D  =  diag 

(d2 

d2  1/d2) 

9 

9 

12 

.0490 

.0565 

.0720 

.0375 

7 

7 

16 

.0450 

.0695 

.0920 

.0435 

8 

11 

11 

.0770 

.0580 

.0715 

.0445 

6 

12 

12 

.0960 

.0650 

.0775 

.0460 

Note.  When  n,  =  n,  *  n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *  nj  -  n3,  the  form  of  contamination  is  I:D:D.  The  underlined  fs 
are  outside  the  interval  a   ±    2ja( l-a)/2000 . 


Table  17 


Negative 

Cor 

dition 

N/p  = 

10.  p 

=  3.  d  ■ 

=  3 ,  and  a  - 

.05 

ni 

"z 

n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  -  d2I 

9 

9 

12 

.1055 

.0885 

.1145 

.0590 

7 

7 

16 

.2495 

.1050 

.1475 

.0580 

8 

11 

11 

.1815 

.0890 

.1075 

.0660 

6 

12 

12 

.2900 

.1305 

.1625 

.0855 

D  =  diag 

(d2 

d2  1) 

9 

9 

12 

.0865 

.0725 

.0985 

.0470 

7 

7 

16 

.1555 

.0860 

.1215 

.0530 

8 

11 

11 

.1345 

.0775 

.0970 

.0495 

6 

12 

12 

.2050 

.1025 

.1295 

.0715 

D  =  diag 

(d2 

d2  1/ 

d? 

9 

9 

12 

.0685 

.0725 

.0895 

.0500 

7 

7 

16 

.1225 

.0910 

.1160 

.0570 

8 

11 

11 

.1145 

.0615 

.0775 

.0430 

6 

12 

12 

.1795 

.0825 

.1010 

.0575 

Note.  When  n,  =  n2  *  n3 ,  the  form  of  contamination  is  D:D:I;  when 
n1  *  ru  =  n3.  the  form  of  contamination  is  D:I:I.  The  underlined  fs 
are  outside  the  interval  a  ±  27a( l-a)/2000 . 
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Table  18 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  10.  p  =  6.  d  =  3.  and  a  = !_05 


ru      n3        Pillai-  Johansen's  James'  James' 

Bartlett     test       1st  order    2nd  order 
trace  test        test 


D  = 

d2I 

18 

18 

24 

.0290 

.0575 

.0750 

.0275 

15 

15 

30 

.0055 

.0685 

.0860 

.0395 

16 

22 

22 

.0310 

.0770 

.0950 

.0375 

12 

24 

24 

.0145 

.0650 

.0740 

.0315 

D  - 

diag 

(d2 

d2  d2 

1  1  1) 

18 

18 

24 

.0340 

.0565 

.0675 

.0285 

15 

15 

30 

.0120 

.0700 

.0840 

.0265 

16 

22 

22 

.0380 

.0545 

.0715 

.0290 

12 

24 

24 

.0285 

.0735 

.0905 

.0335 

D  = 

diag 

(d2 

d2  d2 

1/d2  1/d2 

1/d2) 

18 

18 

24 

.0675 

.0710 

.0880 

.0375 

15 

15 

30 

.0720 

.0795 

.  1000 

.0360 

16 

22 

22 

.1060 

.0625 

.0770 

.0285 

12 

24 

24 

.1680 

.0710 

.0885 

.0345 

Note .  When  n,  =  r^  *  n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  ¥■  n2  =  n3,  the  form  of  contamination  is  I:D:D.  The  underlined  fs 
are  outside  the  interval  a  ±  27a( l-a)/2000 . 
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Table  19 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =10,  p  =  6,  d  =  3,  and  a   =  .05 


r^      n3        Pillai-  Johansen's  James'  James' 

Bartlett     test       1st  order   2nd  order 


D  ■= 

d*I 

18 

18 

24 

.1280 

.0915 

15 

15 

30 

.2650 

.1070 

16 

22 

22 

.1990 

.0875 

12 

24 

24 

.3940 

.1480 

D  = 

diag 

(d2 

d2  d2  ] 

-  1  1) 

18 

18 

24 

.0745 

.0685 

15 

15 

30 

.1365 

.0930 

16 

22 

22 

.1275 

.0720 

12 

24 

24 

.1925 

.0890 

.1155       .0320 


1370       .0385 
1105       .0435 


1705       .0680 


0800       .0275 


1115       .0350 


570       .0355 


.1085       .0405 


D  =  diag(d2  d2  d2  1/d2  1/d2  1/d2) 

18   18   24     .0495       .0590  .0720  .0260 

15  15   30     .0620       .0685  .0870  .0330 

16  22   22     .1095       .0670  .0800  .0325 
12   24   24     .1485       .0620  .0725  .0340 


Note.  When  n,  -  n2  y  n3 ,  the  form  of  contamination  is  D:D:I;  when 
n,  y  ru  =  n3,  the  form  of  contamination  is  D:I:I.  The  underlined  fs 
are  outside  the  interval  a  ±  27q(1-q)/2000 . 
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test  and  the  second  order  test  as  d  increased.  These  frequencies 
are  misleading  for  Johansen's  test:  there  is  a  marked  increase  in  r 
in  the  negative  condition  when  d  changes  from  Jl . 5  to  3. 
Specifically,  the  mean  r    increases  from  .0710  to  .0863. 

When  d  =  Jl . 5 ,  the  estimated  rs  outside  the  criterion  interval 
for  the  Pillai-Bartlett  trace  are  smaller  than  a  -  .05  under 
positive  conditions  (smaller  samples  obtained  from  the  populations 
having  the  contaminated  variance-covariance  matrices)  and  larger 
than  a  under  negative  conditions  (smaller  samples  obtained  from  the 
populations  having  the  contaminated  variance-covariance  matrices). 
This  is  similar  to  the  performance  of  Hotelling's  T2  found  by 
Hakstian,  Roed,  and  Lind  (1971).  The  same  pattern  is  observed  for 
d  =  3.0  except  when  D  =  diag(d2  d2  1/d2)  or  diag(d2  d2  d2  1/d2  1/d2 
1/d2)  .  Then  r  is  near  or  larger  than  a  in  both  positive  and 
negative  conditions.  For  Johansen's  test  and  James'  first  order 
test,  the  rs  outside  the  criterion  interval  tend  to  be  larger  than 
a.  Estimated  type  I  error  rates  for  Johansen's  test  are  generally 
closer  to  a  than  the  rs  for  the  first  order  test  are.  This  is 
similar  to  the  performance  of  Yao's  test  and  James'  first  order  test 
when  p  -  2  (Algina  and  Tang,  1988).  For  James'  second  order  test, 
the  rs  outside  the  criterion  interval  are  smaller  than  a. 

For  d  =  Jl . 5  and  3.0  respectively,  mean  rs  for  various  levels 
in  the  design  are  reported  in  Tables  20  and  21  respectively.  Two 
factors,  sample-size-covariance-matrix  (SSCM)  relationship  and 
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Table  20 

The  Effects  of  Four  Factors  on  the  Means  of  ?s  of  the  Four  Tests: 
d  -  7l~5 


Factor 

Pillai- 

Johansen' s 

James ' 

James ' 

Bartlett 

test 

1st  order 

2nd  order 

trace 

test 

test 

Sample-size- 

covariance- 

matrix 

relationship 

Positive 

.0383 

.0654 

.0826 

.0371 

Negative 

.0577 

.0710 

.0898 

.0401 

ni :  "2 :  n3 

P   /   Na 

1:1:1.3 

.0391/. 0518 

.0587 

.0741 

.0348 

1:1:2 

.0322/. 0603 

.0721 

.0929 

.0387 

1:1.3:1.3 

.0401/. 0542 

.0627 

.0773 

.0367 

1:2:2 

.0421/. 0645 

.0793 

.1004 

.0442 

P 

3 

.0464 

.0672 

.0866 

.0440 

6 

.0496 

.0691 

.0858 

.0332 

D 

d2I 

.0480 

.0666 

.0839 

.0388 

diag(d2  or  1) 

.0478 

.0687 

.0875 

.0390 

diag(d2  or  1/d2) 

.0483 

.0692 

.0872 

.0380 

aThe  mean  ts  under  different  sample  size  ratios  are  presented 
separately  for  positive  condition  (P)  and  negative  condition  (N)  for 
the  Pillai-Bartlett  trace. 
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Table  21 

The  Effects  of  Four  Factors  on  the  Means  of  rs  of  the  Four  Tests: 
d  =  3 


Factor       Pillai-     Johansen's     James'       James' 

Bartlett     test       1st  order    2nd  order 
trace  test        test 


Sample-size- 
covariance- 
matrix 
relationship 

Positive 

.0455 

.0636 

.0796 

.0368 

Negative 

.1575 

.0863 

.1081 

.0476 

n, :  nj :  n3 

P   /   Na 

1:1:1.3 

.0431/. 0854 

.0670 

.0842 

.0371 

1:1:2 

.0256/. 1652 

.0794 

.1032 

.0414 

1:1.3:1. 

3 

.0542/. 1444 

.0700 

.0864 

.0417 

1:2:2 

.0592/. 2349 

.0834 

.1016 

.0487 

P 

3 

.0991 

.0741 

.0949 

.0496 

6 

.1038 

.0758 

.0929 

.0348 

D 

d?I 

.1246 

.0845 

.1059 

.0470 

diag(d2  or 

1) 

.0840 

.0719 

.0906 

.0396 

diag(d  or 

1/d2) 

.0959 

.0686 

.0851 

.0401 

aThe  mean  rs  under  different  sample  size  ratios  are  presented 
separately  for  positive  condition  (P)  and  negative  condition  (N)  for 
the  Pillai-Bartlett  trace. 
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sample  size  ratio,  tended  to  have  larger  effects  than  the  other 
factors  had  on  the  performance  of  the  four  tests.  The  rs  for  all 
four  tests  tend  to  be  larger  in  the  negative  condition  than  in  the 
positive  condition.  For  the  second  order  test  this  trend  resulted 
in  better  performance  in  the  negative  condition.  For  Johansen's 
test  and  the  first  order  test,  it  resulted  in  poorer  performance  in 
the  negative  condition.  For  d  =  Jl . 5 ,  the  Pillai -Bartlett  trace 
performed  better  in  the  negative  condition:  of  the  20  rs  outside 
the  criterion  interval,  only  five  came  from  the  negative  condition. 
The  same  pattern  was  observed  with  d  =  3.0  and  p  =  3.  However,  when 
p  increased  to  6  the  test  performed  more  poorly  in  the  negative 
condition. 

The  ts  for  Johansen's  and  the  first  order  test  tend  to 
increase  as  the  sample  size  ratio  becomes  more  extreme,  resulting 
in  poorer  performance.  For  the  Pillai-Bartlett  trace  the  effect  of 
the  sample  size  ratio  depended  on  the  SSCM  relationship.  With  a 
negative  relationship  the  test  tended  to  become  more  liberal  as  the 
ratio  became  more  extreme.  The  effect  was  larger  for  d  =  3.0  than 
it  was  for  d  =  Jl. 5.  With  a  positive  relationship,  the  effect  of 
the  sample  size  ratio  was  quite  complicated,  depending  on  d,  p,  and 
the  form  of  D.  For  James'  second  order  test,  there  is  a  tendency 
for  t  to  increase  as  the  sample  size  ratio  becomes  more  extreme. 
This  increase  tended  to  result  in  better  performance  for  the  test. 


6  5 
Tvpe  I  error  rates  appear  to  be  unaffected  by  variation  in  p 
for  the  Pillai-Bartlett  trace,  Johansen's  test,  and  James'  first 
order  test.  James'  second  order  test  was  affected  negatively  by  an 
increase  in  p .  In  fact,  when  d  -  J\ .  5 ,  the  poorer  performance  of 
the  second  order  test,  in  comparison  to  the  Pillai-Bartlett  trace 
was  due  to  its  performance  when  p  =  6.  For  p  -  3,  the  second  order 
test  has  nine  rs  outside  the  criterion  interval.  For  p  =  6,  the 
number  of  rs  outside  the  criterion  interval  increases  to  20. 

The  form  of  D  had  little  effect  on  rs  for  the  tests  except  for 
the  Pillai-Bartlett  trace  when  d  =  3.0.  Then  the  test  performed 
better  when  D  =  diag(d2  d2  1/d2)  and  D  =  diag(d2  d2  d2  1/d2  1/d2  1/d2)  . 
For  the  contaminated  covariance  matrices  with  these  forms,  only  one 
of  16  rs  is  outside  the  criterion  interval. 
The  Performance  of  the  Four  Tests  When  N/p  =  15 

Estimated  rs  for  the  Pillai-Bartlett  trace,  Johansen's  test, 
James'  first  order  test,  and  James'  second  order  test  for  conditions 
in  which  N/p  =  15  and  d  =  Jl . 5  are  reported  in  Tables  22,  23,  24, 
and  25.  Forty-eight  rs  are  reported  for  each  test.  For  Pillai- 
Bartlett  trace,  24  rs  are  outside  the  criterion  interval.  For 
Johansen's  test,  13  rs  are  outside  the  criterion  interval.  For 
James'  first  order  test,  37  rs  are  outside  the  criterion  interval. 
For  James'  second  order  test,  15  rs  are  outside  the  criterion 
interval.   Given  d  =  71 . 5 ,  as  N/p  increases  from  10  to  15,  the 
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Table  22 


•our  Mui  1 


-  onansei 


11 

11 

9  ^ 

.0290 

13 

16 

16 

.0410 

9 

18 

IS 

.0390 

D  - 

■  diag 

(d2 

d2  1) 

13 

13 

19 

.0380 

11 

11 

23 

.0315 

13 

16 

16 

.0350 

9 

18 

18 

.0480 

James '       James ' 
1st  order    2nd  orde: 


03^5        .0520  .0610  .0415 

0510  .0565  .0445 

0495  .0555  .0420 

.0565  .0665  .0420 

.0360 


0470 

.0565 

0440 

.0560 

0460 

.0555 

0585 

.0710 

.0355 


0395 


0455 


13  13  19  .0-405  .0500  .0530  .^00 

11  11  23  .0435  .0580  .0705  .0455 

13  16  16  .0495  .0550  .0625  .0450 

9  18  18  .0410  .0530  .0635  .0430 


Note .  When  n.  -  n,  *  n3 ,  the  form  of  contamination  is  1:1:0;  when 
n  *  ru  =  n3.  the  form  of  contamination  is  I:D:D.  The  underlined  fs 
are  outside  the  interval  a   ±  2ja( 1-q) /2000 . 
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Table  23 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests 


Negative  Condition,  N/p  =15,  p  =  3.  d  =  71 . 5 .  and  a 


05 


r^   n3    Pillai-     Johansen's     James'       James' 

Bartlett     test       1st  order    2nd  order 
trace  test        test 


D  - 

d2I 

13 

13 

19 

.0715 

.0620 

.0715 

.0535 

11 

11 

23 

.0785 

.0555 

.0675 

.0420 

13 

16 

16 

.0650 

.0555 

.0605 

.0475 

9 

18 

18 

.0665 

.0570 

.0685 

.0455 

D  - 

diag 

(d2 

d2  1) 

13 

13 

19 

.0610 

.0550 

.0635 

.0460 

11 

11 

23 

.0630 

.0545 

.0670 

.0405 

13 

16 

16 

.0510 

.0640 

.0690 

.0495 

9 

18 

18 

.0570 

.0555 

.0635 

.0405 

D  - 

diag 

(d2 

d2  1/d2) 

13 

13 

19 

.0565 

.0570 

.0660 

.0480 

11 

11 

23 

.0600 

.0630 

.0735 

.0505 

13 

16 

16 

.0530 

.0550 

.0635 

.0435 

9 

18 

18 

.0530 

.0635 

.0705 

.0500 

Note .  When  n,  =  n2  *  n3,  the  form  of  contamination  is  D:D:I;  when 
n,  *  r^  =  n3,  the  form  of  contamination  is  D:I:I.  The  underlined  fs 
are  outside  the  interval  a  ±  2ja( l-a)/2000 . 


Table  24 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  15,  p  =  6.  d  =  7i~3 .  and  a  =  .05 


n.      ru      n3        Pillai-  Johansen's  James'  James' 

Bartlett     test       1st  order   2nd  order 
trace  test        test 


.0345 

.0445 
.0585 

.0480 
.0660 

.0315 

.0275 

.0425 

.0445 

.0545 

.0625 

.0415 

.0310 

.0605 

.0695 

.0400 

D  -  d2I 

27  27  36 

22  22  46 

26  32  32 
18  36  36 

D  =  diag(d2  d2  d2  1  1  1) 

27  27  36  .0365  .0540        .0610       .0400 
22  22  46  .0345       .0570        .0620       .0365 

26  32  32  .0545  .0685        .0760       .0510 
18  36  36  .0400       .0555        .0620       .0375 

D  =  diag(d2  d2  d2  1/d2   1/d2   1/d2) 

27  27  36  .0420  .0520 
22  22  46  .0470  .0540 
26  32  32  .0465  .0470 
18  36  36  .0580       .0655 


.0575 

.0345 

.0605 

.0340 

.0550 

.0320 

.0765 

.0415 

Note .  When  n,  -  n,  r1  n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *  r^  =  n3,  the  form  of  contamination  is  I:D:D.  The  underlined  fs 
are  outside  the  interval  a  ±  2ja( l-a)/2000 . 


(/> 


Table  25 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =  15.  p  =  6.  d  =  7l .  5  ,  and  a   = J15 


ru   n3    Pillai-     Johansen's     James'       James' 

Bartlett     test       1st  order   2nd  order 
trace  test        test 


D  =   diag(d2  d2  d2  1/d2   1/d2   1/d2) 

27   27   36  .0530       .0560 

22   22   46  .0540       .0590 

26   32   32  .0405       .0530 

18   36   36  .0540       .0645 


D  -  d2I 

27 

27 

36 

.0485 

.0420 

.0525 

.0280 

22 

22 

46 

.0870 

.0710 

.0820 

.0415 

26 

32 

32 

.0565 

.0640 

.0690 

.0465 

18 

36 

36 

.0820 

.0735 

.0815 

.0455 

D  =  diag 

(d2 

d2  d2  ] 

-  1  1) 

27 

27 

36 

.0610 

.0570 

.0660 

.0435 

22 

22 

46 

.0730 

.0630 

.0765 

.0420 

26 

32 

32 

.0565 

.0510 

.0600 

.0330 

18 

36 

36 

.0645 

.0655 

.0750 

.0415 

.0660 

.0395 

.0680 

.0405 

.0575 

.0385 

.0700 

.0420 

Note.  When  n,  =  ru  *  n3 ,  the  form  of  contamination  is  D:D:I;  when 
nf  ^  r^  -  n3,  the  form  of  contamination  is  D:I:I.  The  underlined  fs 
are  outside  the  interval  a  ±  2ja( 1-q)/2000 . 
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number  of  ?s  outside  the  criterion  interval  decreases  for  all  tests 
except  the  Pillai-Bartlett  trace.  There  is  a  65%,  23%,  and  48% 
decrease  for  Johansen's  test,  James'  first  order  test,  and  James' 
second  order  test  respectively. 

The  results  for  d  =  3.0  are  reported  in  Tables  26  through  29. 
For  the  Pillai-Bartlett  trace,  all  but  five  rs  are  outside  the 
criterion  interval.  For  Johansen's  test,  21  ts  are  outside  the 
criterion  interval.  For  James  first  order  test,  38  ts  are  outside 
the  criterion  interval.  For  James'  second  order  test,  23  ts  are 
outside  the  criterion  interval.  As  in  d  -  J1.5 ,  the  number  of  ts 
outside  the  criterion  interval  decreases  for  all  tests  except  the 
Pillai-Bartlett  trace  as  N/p  increases  from  10  to  15.  There  is  a 
47%,  21%,  and  22%  decrease  for  Johansen's  test,  James'  first  order 
test,  and  James'  second  order  test  respectively.  These  percentage 
decreases  as  N/p  increases  are  smaller  for  d  =  3  than  the 
corresponding  decreases  were  for  d  -  JT75 .  Given  N/p  -  15,  when  d 
increases  from  /l . 5  to  3,  there  is  a  87%,  54%,  and  60%  increase  in 
the  number  of  ts  outside  the  criterion  interval  for  the  Pillai- 
Bartlett  trace,  Johansen's  test,  and  James'  second  order  test.  For 
James'  first  order  test,  the  increase  is  trivial. 

For  d  =  /l . 5  the  ts  for  all  four  tests  tend  to  be  larger  in 
the  negative  condition  than  in  the  positive  condition.   For  the 
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Table  26 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  15,  p  =  3,  d  -  3,  and  a  -=  .05 


nj      n3        Pillai-  Johansen's  James'  James' 

Bartlett     test       1st  order   2nd  order 
trace  test        test 


D  -=  d2I 

13 

13 

19 

.0220 

.0525 

.0585 

.0405 

11 

11 

23 

.0120 

.0600 

.0690 

.0510 

13 

16 

16 

.0440 

.0580 

.0665 

.0485 

9 

18 

18 

.0250 

.0570 

.0685 

.0470 

D  =  diag 

(d2 

d2  1) 

13 

13 

19 

.0350 

.0530 

.0630 

.0445 

11 

11 

23 

.0195 

.0590 

.0700 

.0465 

13 

16 

16 

.0485 

.0580 

.0655 

.0490 

9 

18 

18 

.0310 

.0530 

.0585 

.0445 

D  -  diag 

(d2 

d2  1/d2) 

13 

13 

19 

.0550 

.0685 

.0800 

.0585 

11 

11 

23 

.0480 

.0580 

.0665 

.0435 

13 

16 

16 

.0745 

.0595 

.0675 

.0475 

9 

18 

18 

.0935 

.0440 

.0510 

.0345 

Note .  When  n,  =  n2  x  n3,  the  form  of  contamination  is  I:I:D;  when 
n,  r*  n,  =  n3,  the  form  of  contamination  is  I:D:D.  The  underlined  fs 
are  outside  the  interval  a   ±    2ja(l-o) /2000 . 
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Table  27 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =15.  p  =  3.  d  ^  3,  and  a   =  .05 


n.   ru  n3    Pillai-     Johansen's     James'       James' 

Bartlett      test       1st  order    2nd  order 
trace  test        test 


D  = 

d2I 

13 

13 

19 

.1110 

.0590 

.0695 

.0410 

11 

11 

23 

.1975 

.0750 

.0910 

.0470 

13 

16 

16 

.1400 

.0650 

.0740 

.0535 

9 

18 

18 

.2670 

.0855 

.0965 

.0555 

D  = 

diag 

(d2 

d2  1) 

13 

13 

19 

.0875 

.0675 

.0745 

.0500 

11 

11 

23 

.1430 

.0620 

.0795 

.0415 

13 

16 

16 

.0980 

.0520 

.0580 

.0420 

9 

18 

18 

.1955 

.0685 

.0800 

.0480 

D  = 

diag 

(d2 

d2  1/d2) 

13 

13 

19 

.0705 

.0545 

.0670 

.0435 

11 

11 

23 

.0960 

.0615 

.0765 

.0475 

13 

16 

16 

.1035 

.0530 

.0605 

.0455 

9 

18 

18 

.1935 

.0640 

.0765 

.0500 

Note .  When  n,  -  n2  *  n3 ,  the  form  of  contamination  is  D:D:I;  when 
n,  *  r^  =  n3,  the  form  of  contamination  is  D:I:I.  The  underlined  fs 
are  outside  the  interval  a  ±  27q(1-q)/2000 . 
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Table  28 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  15,  p  •=  6  .  d  -  3.  and  a  =  .05 


n.      ru      n3        Pillai-  Johansen's  James'  James' 

Bartlett     test       1st  order   2nd  order 
trace  test        test 


.0330 


D  - 

d2I 

27 

27 

36 

.0360 

.0455 

.0555 

22 

22 

46 

.0025 

.0530 

.0595 

26 

32 

32 

.0370 

.0550 

.0620 

18 

36 

36 

.0150 

.0565 

.0625 

D  - 

diag 

(d2 

d2  d2  1  1  1) 

27 

27 

36 

.0325 

.0465 

.0535 

22 

22 

46 

.0150 

.0515 

.0595 

26 

32 

32 

.0465 

.0555 

.0625 

18 

36 

36 

.0300 

.0530 

.0605 

D  =  diag(d2  d2  d2  1/d2   1/d2   1/d2) 

27   27   36  .0600       .0480 

22   22   46  .0745       .0520 

26   32   32  .0850       .0495 

18   36   36  .1605       .0555 


.0540 
.0605 
.0585 
.0650 


.0325 


0370 


0390 


0310 


.0325 


.0395 


.0360 


.0305 


.0340 


.0350 


.0375 


Note.   When  n,  =  n2   *  n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *   n,  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined  fs 


ire  outside  the  interval  a   ±  27q(1-q)/2000 . 
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Table  29 

Actual  Tvpe  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =  15,  p  =  6,  d=3.  and  a  =  .05 


n.   n,  n3   Pillai-    Johansen's    James'      James' 

Bartlett      test       1st  order    2nd  order 


trace 

D  - 

-  d2I 

27 

27 

36 

.1135 

.0600 

22 

22 

46 

.2745 

.0600 

26 

32 

32 

.1555 

.0570 

18 

36 

36 

.3810 

.0740 

test        test 


.0710  .0345 

.0770  .0290 


.0650  .0395 

.0880  .0390 


D  -   diag(d2   d2   d2  1    1    1) 

27   27   36  .0800  .0605  .0660  .0395 

22   22   46  .1415  .0655  .0765  .0325 

26  32  32  .1045  .0650  .0690  .0405 
18   36   36  .2050  .0645  .0760  .0385 

D  =  diag(d2  d2  d2  1/d2  1/d2  1/d2) 

27  27  36  .0630  .0505  .0615  .0345 
22  22  46  .0755  .0625  .0720  .0380 
26  32  32  .0975  .0600  .0690  .0430 
18   36   36  .1715  .0645  .0690  .0460 


Note .  When  n,  =  r^  H  n3 ,  the  form  of  contamination  is  D:D:I;  when 
n,  y  n,  =  n3.  the  form  of  contamination  is  D:I:I.  The  underlined  fs 
are  outside  the  interval  a  ±  2ja{ l-a)/2000 . 
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second  order  test  this  trend  resulted  in  better  performance  in  the 
negative  condition.  For  Johansen's  test  and  the  first  order  test 
it  resulted  in  better  performance  in  the  positive  condition.  In  the 
positive  condition,  Johansen's  test  has  three  of  24  rs  outside  the 
criterion  interval.  In  the  negative  condition,  James'  second  order 
test  has  three  of  24  rs  outside  the  criterion  interval.  For  the 
Pillai-Bartlett  trace,  performance  was  poor  under  both  positive  and 
negative  conditions.  In  the  positive  condition,  the  test  tended  to 
yield  rs  much  smaller  than  a;  in  the  negative  condition,  the  test 
tended  to  yield  rs  much  larger  than  a.  However,  when  D  -  diag(d2  d2 
1/d2)  or  D  -  diag(d2  d2  d2  1/d2  1/d2  1/d2) ,  the  test  tended  to  be 
liberal  in  both  the  positive  and  negative  conditions.  With  one 
exception,  the  effect  of  the  SSCM  relationship  on  the  tests  was 
similar  when  d  -  3.0  to  its  effect  when  d  -  JTTS:  this  factor  did 
not  appear  to  affect  r  for  the  second  order  test  when  d  -  3.0. 

Unlike  the  conditions  in  which  N/p  -  10,  changing  the  sample 
size  ratio  did  not  have  a  clear  impact  on  the  performance  of  the 
tests.  For  d  -  3.0,  when  the  sample  size  ratio  becomes  more 
extreme,  the  rs  for  Johansen's  test  and  James  first  order  test 
increase,  which  leads  to  poorer  performance  of  tests.  However,  the 
impact  of  this  factor  was  smaller  than  that  of  SSCM  relationship. 
The  sample  size  ratio  did  not  have  an  effect  on  James'  second  order 
test.    For  the  Pillai-Bartlett  trace,  as  the  sample  size  ratio 
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becomes  more  extreme,  the  test  became  more  liberal  in  the  negative 
condition  and  more  conservative  in  the  positive  condition  except 
when  D  had  the  form  of  diag(d2  d2  1/d2)  or  diag(d2  d2  d2  1/d2  1/d2 
1/d2)  .  With  this  form  for  D,  the  Pillai -Bartlett  trace  tended  to 
become  more  liberal,  in  both  the  positive  and  the  negative 
condition,  as  the  sample  size  ratio  became  more  extreme. 

For  d  -  J\ .  5 ,  both  Johansen's  test  and  the  second  order  test 
performed  better  when  p  -  3  than  when  p  -  6.  When  p  -  3,  both  of 
them  have  four  rs  outside  the  criterion  interval.  When  p  increases 
to  6,  there  are  nine  rs  outside  the  criterion  interval  for 
Johansen's  test  and  11  for  the  second  order  test.  For  the  Pillai - 
Bartlett  trace  and  the  first  order  test,  variation  in  p  did  not  have 
an  impact  on  the  performance  of  the  tests  when  d  =  JTTb  .  For  James' 
second  order  test,  when  d  increased  to  3.0,  the  effect  of  the  number 
of  variables  (p)  became  even  stronger  than  when  d  =  JT7S .  When  p 
=  3,  the  rs  for  the  second  order  test  are  close  to  a  both  in  the 
positive  and  negative  condition.  Only  two  rs  are  outside  the 
criterion  interval.  However,  when  p  increases  to  6,  the  rs  are 
smaller  than  a  in  both  conditions;  only  three  of  2U  rs  are  in  the 
criterion  interval.  The  number  of  variables  (p)  did  not  have  strong 
impact  on  the  other  tests  when  d  =  3.0. 

For  the  Pillai-Bartlett  trace,  the  effect  of  the  form  of  D 
when  d  -  7l . 5  was  similar  to  the  effect  observed  when  N/p  -  10  and 
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a2    A2 


d  -  JTTb .  The  test  performed  better  with  D  -  diag(d  d  1/d  )  or  D 
=  diag(d2  d2  d2  1/d2  1/d2  1/d2)  than  with  D  in  the  other  forms.  With 
contaminated  covariance  matrices  of  these  forms,  only  one  r  of  the 
16  estimated  for  the  Pillai-Bartlett  trace  is  outside  the  criterion 
interval.  When  d  increases  to  3.0,  the  nature  of  the  effect  is  that 
when  D  =  diag(d2  d2  1/d2)  or  D  =  diag(d2  d2  d2  1/d2  1/d2  1/d2),  the 
Pillai-Bartlett  trace  tended  to  perform  better  in  the  positive 
condition  than  it  did  in  the  negative  condition.  The  form  of  D  did 
not  effect  the  other  tests. 

In  general,  when  N/p  -  15,  Johansen's  test  and  James'  second 
order  test  performed  better  than  either  the  Pillai-Bartlett  trace 
or  James'  first  order  test.  When  judged  in  terms  of  the  number  of 
ts  outside  the  criterion  interval  Johansen's  test  performed  well  in 
the  positive  conditions,  and  except  for  conditions  in  which  p  -  6 
and  d  =  3,  the  second  order  test  performed  well  in  the  negative 
conditions.  In  the  exceptional  condition,  both  the  second  order 
test  and  Johansen's  test  had  the  vast  majority  of  rs  outside  the 
criterion  interval.  However,  the  average  t  for  Johansen's  test  was 
.062  and  for  the  second  order  test  was  .038.  The  ts  for  Johansen's 
test  were  not  too  extreme,  even  in  the  negative  condition,  the 
condition  in  which  Johansen's  test  performed  more  poorly  .  Of  48 
ts,  five  were  larger  than  .07  and  only  one  was  larger  than  .08. 
Similarly,  rs  for  the  second  order  test  were  not  too  extreme  in  the 
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positive  condition.  None  of  these  rs  was  smaller  than  .03.  Since  r 
for  each  test  is  reasonably  near  a  in  principle,  either  test  might 
be  used.  However,  in  practice,  programming  the  second  order  test 
is  substantially  more  complicated  than  programming  Johansen's  test. 
The  Performance  of  the  Four  Tests  When  N/p  -  20 

Estimated  rs  for  the  Pillai-Bartlett  trace,  Johansen's  test, 
James'  first  order  test,  and  James'  second  order  test  in  conditions 
in  which  N/p  =  20  and  d  =  71. 5  are  reported  in  Tables  30  through  33. 
Forty-eight  rs  are  reported  of  each  test.  For  the  Pillai-Bartlett 
trace,  19  fs  are  outside  the  criterion  interval.  For  Johansen's 
test  three  rs  are  outside  the  criterion  interval,  all  of  them  in  the 
p  =  6  condition.  For  James'  first  order  test,  18  rs  are  outside  the 
criterion  interval.  For  James'  second  order  test,  13  rs  are  outside 
the  criterion  interval.  Given  d  =  71.5,  all  tests  had  a  notable 
decrease  in  the  number  of  fs  outside  the  criterion  interval  as  N/p 
increased  from  15  to  20.  There  were  21%,  77%,  51%,  and  13% 
decreases  for  Pillai-Bartlett  trace,  Johansen's  test,  James'  first 
order  test,  and  James'  second  order  test  respectively. 

Estimated  rs  for  the  Pillai-Bartlett  trace,  Johansen's  test, 
James'  first  order  test,  and  James'  second  order  test  in  conditions 
in  which  N/p  -  20  and  d  =  3  are  reported  in  Tables  34,  35,  36,  and 
37.  Forty-eight  fs  are  reported  for  each  test.  For  the  Pillai- 
Bartlett  trace,  42  fs  fall  outside  the  criterion  interval.   For 
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Johansen's  test,  seven  rs  fall  outside  the  criterion  interval.  For 
James'  first  order  test,  26  rs  fall  outside  the  criterion  interval. 
For  James'  second  order  test,  nine  rs  fall  outside  the  criterion 
interval.  Except  for  the  Pillai -Bartlett  trace,  there  is  a  decrease 
in  the  number  of  rs  outside  the  criterion  interval  as  N/p  increases 
from  15  to  20.  There  is  a  65%,  31%,  and  62%  decrease  for  Johansen's 
test,  James'  first  order  test,  and  James'  second  order  test 
respectively.  Given  N/p  =20,  as  d  increases  from  Jl . 5  to  3,  the 
number  of  rs  outside  the  criterion  interval  increases  121%,  133%, 
and  44%  for  the  Pillai-Bartlett  trace,  Johansen's  test,  and  James' 
first  order  test  respectively.  For  the  James'  second  order  test, 
there  is  a  31%  decrease. 

The  SSCM  relationship  affected  the  performance  of  all  tests 
except  James'  second  order  test.  The  latter  test  performed  almost 
equally  well  in  both  positive  and  negative  conditions.  There  are 
13  rs  outside  the  criterion  interval  for  the  positive  condition  and 
nine  for  the  negative  condition.  For  the  other  tests,  the  number 
of  rs  outside  the  criterion  interval  is  larger  in  the  negative 
condition  than  in  the  positive  condition.  This  resulted  in  a  better 
performance  for  the  first  order  test  in  the  positive  condition. 
Johansen's  test  performed  better  in  the  positive  condition  except 
when  d  =  71. 5.   When  d  =  Jl . 5  it  performed  well  in  both  conditions. 
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Table  30 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition.  N/p  =20,  p  =  3.  d  =  ./! .  5 .  and  a   -  .05 


ru      n3        Pillai-  Johansen's  James'  James' 

Bartlett     test       1st  order   2nd  order 
trace  test        test 


D  -  d2I 

18 

18 

24 

.0340 

.0440 

.0490 

.0395 

15 

15 

30 

.0320 

.0450 

.0515 

.0415 

16 

22 

22 

.0415 

.0480 

.0515 

.0435 

12 

24 

24 

.0335 

.0470 

.0510 

.0395 

D  -  diag 

(d2 

d2  1) 

18 

18 

24 

.0510 

.0550 

.0575 

.0485 

15 

15 

30 

.0295 

.0540 

.0585 

.0470 

16 

22 

22 

.0450 

.0505 

.0550 

.0435 

12 

24 

24 

.0365 

.0520 

.0580 

.0435 

D  -  diag 

(d2 

d2  1/d2) 

18 

18 

24 

.0500 

.0545 

.0605 

.0510 

15 

15 

30 

.0420 

.0520 

.0600 

.0480 

16 

22 

22 

.0515 

.0500 

.0545 

.0460 

12 

24 

24 

.0400 

.0485 

.0560 

.0410 

Note.  When  n,  =  r^  *  n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *  ru  -  n3,  the  form  of  contamination  is  I:D:D.  The  underlined  fs 
are  outside  the  interval  a   ±    2ja( l-a)/2000 . 
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Table  31 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Neaative  Condition.  N/p  =  20.  p  ~  3.  d  =  ./!  .  5 .  and  a   -  .05 


n,  n3   Pillai-    Johansen's    James'      James' 

Bartlett     test       1st  order    2nd  order 
trace  test        test 


D  - 

d2I 

18 

18 

24 

.0560 

15 

15 

30 

.0675 

16 

22 

22 

.0620 

12 

24 

24 

.0640 

D  = 

diag 

(d2 

d2  1) 

18 

18 

24 

.0595 

15 

15 

30 

.0615 

16 

22 

22 

.0490 

12 

24 

24 

.0590 

D  - 

diag 

(d2 

d2  1/d2) 

18 

18 

24 

.0470 

15 

15 

30 

.0515 

16 

22 

22 

.0575 

12 

24 

24 

.0535 

.0475  .0525  .0430 

.0500  .0560  .0405 

.0560  .0625  .0500 

.0515  .0620  .0420 


.0545 
.0595 
.0485 
.0530 


.0600 

.0465 

.0645 

.0530 

.0545 

.0405 

.0610 

.0450 

.0480 

.0500 

.0410 

.0470 

.0535 

.0380 

.0545 

.0610 

.0500 

.0520 

.0600 

.0440 

Note.  When  n1  =  r^  ¥■  n3,  the  form  of  contamination  is  D:D:I;  when 
n,  ^  r^,  =  n3,  the  form  of  contamination  is  D:I:I.  The  underlined  fs 
are  outside  the  interval  a   ±   2ja(l-a) /2000 . 
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Table  32 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition.  N/p  °  20.  p  =  6.  d  =  ./TT3 .  and  q  -  .05 


James ' 


n1      n,     n3        Pillai-  Johansen's  James' 

Bartlett     test       1st  order    2nd  order 
trace  test         test 


D  =  diag(d2  d2  d2  1/d2  1/d2  1/d2) 

36  36  48  .0535  .0475 
30  30  60  .0520  .0585 
32  44  44  .0490  .0490 
24   48   48     .0565        .0585 


.0565 
.0655 
.0550 
.0645 


D  -  d2I 

36 

36 

48 

.0370 

.0450 

.0480 

.0355 

30 

30 

60 

.0315 

.0600 

.0660 

.0445 

32 

44 

44 

.0350 

.0465 

.0495 

.0385 

24 

48 

48 

.0275 

.0475 

.0530 

.0360 

D  =  diag 

(d2 

d2  d2  1  1  1) 

36 

36 

48 

.0405 

.0515 

.0555 

.0415 

30 

30 

60 

.0340 

.0520 

.0590 

.0410 

32 

44 

44 

.0470 

.0525 

.0545 

.0475 

24 

48 

48 

.0415 

.0590 

.0675 

.0435 

.0380 
.0410 
.0385 
.0445 


Note.  When  n,  =  r^  *  n3,  the  form  of  contamination  is  I:I:D;  when 
n  *  rL,  -  n3,  the  form  of  contamination  is  I :  D :  D .  The  underlined  fs 
are  outside  the  interval  a   ±  27a(l-a)/2000 . 
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Table  33 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition.  N/p  =  20 .  p  =    6 .  d  =  ./l .  5 ,  and  a  =  .05 


n,      n,     n3        Pillai-  Johansen's  James' 


James ' 


lartlett  test  1st   order        2nd   order 

test  test 


0510 

.0470 

.0515 

.0400 

.0805 

.0560 

.0625 

.0425 

.0520 

.0415 

.0485 

.0310 

.0815 

.0635 

.0670 

.0460 

D  -   d2I 

36  36  48 

30  30  60 

32  44  44 

24  48  48 

D  =   diag(d2  d2  d2  1    1    1) 

36  36  48  .0470       .0505        .0550       .0420 

30  30  60  .0635       .0530        .0580       -0385 

32  44  44  .0590       .0505        .0520       .0430 

24  48  48  .0690                  .0520                    .0590                  .0400 

D  =   diag(d2  d2  d2  1/d2   1/d2   1/d2) 

36  36  48  .0460 

30  30  60  .0555 

32  44  44  .0530 

24  48  48  .0550 


.0505 

.0570 

.0420 

.0630 

.0725 

.0455 

.0590 

.0630 

.0495 

.0555 

.0645 

.0410 

Note.  When  n,  -  r^  *  n3,  the  form  of  contamination  is  D : D : I ;  when 
n  *  r^  -  n3,  the  form  of  contamination  is  D:I:I.  The  underlined  fs 
are  outside  the  interval  a   ±    2ja(l-a) /2000 . 


8'. 


Table  34 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition.  N/p  -  20.  p  =  3.  d  =  3,  and  a  -    .05 


trace 


ru   n3    Pillai-     Johansen's     James'       James' 

Bartlett     test       1st  order   2nd  order 


test        test 


.0535 

.0555 

.0475 

.0540 

.0585 

.0480 

.0515 

.0580 

.0475 

.0550 

.0585 

.0505 

D  -  d2I 

18  18  24  .0380 

15  15  30  .0125 

16  22  22  .0360 
12  24  24  .0275 

D  =  diag(d2  d2  1) 

18  18  24  .0365        .0480       .0530       .0420 

15  15  30  .0165         .0570        .0600        .0505 

16  22  22  .0445        .0535       .0575       .0475 
12  24  24  .0305        .0555       .0620       .0510 

D  =  diag(d2  d2  1/d2) 

18  18  24  .0490 

15  15  30  .0445 

16  22  22  .0755 
12  24  24  .0920 


.0540 

.0570 

.0480 

.0565 

.0630 

.0485 

.0510 

.0575 

.0455 

.0530 

.0595 

.0475 

Note.  When  11,-0,  *  n3 ,  the  form  of  contamination  is  I :  I :  D ;  when 
n  *  n2  -  n3,  the  form  of  contamination  is  I : D : D .  The  underlined  rs 
are  outside  the  interval  a  ±   2ja(l-a)/2000 . 
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Table  35 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition.  N/p  =  20,  p  -  3.  d  -   3.  and  a  =  .05 


Johansen's     James' 


James ' 


r^  n3   Pillai- 

Bartlett     test       1st  order   2nd  order 
trace  test        test 


D  - 

d2I 

18 

18 

24 

0995 

.0540 

.0615 

.0455 

15 

15 

30 

.1680 

.0555 

.0630 

.0435 

16 

22 

22 

.1630 

.0540 

.0600 

.0485 

12 

24 

24 

.2845 

.0635 

.0720 

.0500 

D  = 

diag 

(d2 

d2  1) 

18 

18 

24 

.0680 

.0445 

.0490 

.0355 

15 

15 

30 

.1280 

.0640 

.0730 

.0510 

16 

22 

22 

.1330 

.0605 

.0665 

.0550 

12 

24 

24 

.1990 

.0565 

.0655 

.0440 

D  = 

diag 

(d2 

d2  1/d2) 

18 

18 

24 

.0695 

.0575 

.0615 

.0505 

15 

15 

30 

.0960 

.0540 

.0600 

.0430 

16 

22 

22 

.1090 

.0580 

.0645 

.0500 

12 

24 

24 

.1740 

.0555 

.0620 

.0450 

Note.  When  n,  =  rv,  +  n3,  the  form  of  contamination  is  D:D:I;  when 
n,  *  n^  -  n3,  the  form  of  contamination  is  D:I:I.  The  underlined  fs 
are  outside  the  interval  a   ±  2ja(l-a) /20QG . 
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Table  36 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition.  N/p  =  20.  p  -  6.  d  -  3.  and  a  =  .05 


ru  n3   Pillai-    Johansen's    James'      James' 

Bartlett     test       1st  order   ,2nd  order 


trace 


test        test 


D  -  d2I 

36 

36   48 

.0330 

.0500 

.0550 

.0400 

30 

30   60 

.0025 

.0450 

.0480 

.0375 

32 

44   44 

.0335 

.0510 

.0565 

.0420 

24 

48   48 

.0155 

.0535 

.0610 

.0465 

D  -  diag 

(d2  d2  d2 

1  1  1) 

36 

36   48 

.0390 

.0520 

.0575 

.0425 

30 

30   60 

.0170 

.0485 

.0535 

.0390 

32 

44   44 

.0410 

.0485 

.0535 

.0420 

24 

48   48 

.0205 

.0410 

.0445 

.0350 

D  =  diag 

(d2  d2  d2 

1/d2  1/d2 

1/d2) 

36 

36   48 

.0545 

.0465 

.0515 

.0375 

30 

30   60 

.0730 

.0605 

.0665 

.0510 

32 

44   44 

.1155 

.0580 

.0635 

.0480 

24 

48   48 

.1795 

.0555 

.0620 

.0405 

Note.  When  n,  -  ru,  *  n3,  the  form  of  contamination  is  I:I:D;  when 
n  *  n,  -  n3,  the  form  of  contamination  is  I:D:D.  The  underlined  fs 
are  outside  the  interval  a  ±  2,/a(l-tt)/2000. 
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Table  37 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =  20.  p  -  6,  d  =  3,  and  a  =  .05 


n,   iv,  n3    Pillai-     Johansen's     James' 


James ' 


artlett  test  1st   order        2nd  order 


trace 


.1230 

.0565 

.2560 

.0730 

.1915 

.0565 

.3790 

.0705 

D   -   d2I 

36  36  48 

30  30  60 

32  44  44 

24  48  48 

D   -   diag(d2  d2   d2    1    1    1) 

36  36  48     .0790 

30  30  60     .1345 

32  44  44     .1175 

24  48  48     .2085 

D  =  diag(d2  d2  d2  1/d2  1/d2  1/d2) 

36  36  48 

30  30  60 

32  44  44 

24  48  48 


test        test 


.0655 

.0445 

.0835 

.0440 

.0625 

.0440 

.0760 

.0490 

.0575 

.0620 

.0420 

.0535 

.0625 

.0375 

.0530 

.0590 

.0425 

.0655 

.0735 

.0475 

.0590 

.0505 

.0535 

.0400 

.0730 

.0510 

.0605 

.0390 

.1115 

.0530 

.0575 

.0455 

.1765 

.0520 

.0550 

.0410 

Note.  When  n,  =  r^  *  n3,  the  form  of  contamination  is  D:D:I;  when 
n  *  rv,  =  n3,  the  form  of  contamination  is  D:I:I.  The  underlined  fs 
are  outside  the  interval  a  ±  2ja( l-a)/2000 . 
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For  the  Pillai -Bartlett  trace,  fs  in  the  positive  condition  tend  to 
be  much  smaller  than  a;  rs  in  the  negative  condition  tend  to  be  much 
larger  than  a,  especially  when  sample  size  ratio  is  more  extreme. 
However,  this  trend  does  not  hold  when  D  is  in  the  form  of  diag(d 
d2  1/d2)  or  diag(d2  d2  d2  1/d2  1/d2  1/d2)  for  the  Pillai-Bartlett 
trace.  When  D  is  in  those  forms,  ts  are  in  the  criterion  interval 
for  d  -  JTTS  and  fs  tend  to  be  larger  than  a  for  d  =  3.0,  in  both 
the  positive  and  negative  condition. 

More  extreme  sample  size  ratio  resulted  in  poorer  performance 
of  the  Pillai-Bartlett  trace  and  James'  first  order  test.  When  the 
sample  size  ratio  is  extreme,  the  Pillai-Bartlett  trace  has  smaller 
fs  in  the  positive  condition  and  larger  ts  in  the  negative 
condition.  For  James'  first  order  test,  the  extreme  ratio  leads  to 
larger  fs.  This  factor  did  not  influence  Johansen's  test  or  the 
second  order  test. 

The  number  of  variables  affected  only  the  second  order  test. 
The  fs  tended  to  get  smaller  as  p  increased.  Given  N/p  -  20,  when 
p  =  3  only  four  fs  are  outside  the  criterion  interval.  However, 
when  p  =  6,  18  fs  are  outside  the  criterion  interval. 

In  general,  both  the  Pillai-Bartlett  trace  and  James'  first 
order  test  have  too  many  fs  outside  the  criterion  interval  in 
comparison  with  Johansen's  test  and  the  second  order  test.  They 
are  not  recommended  for  use  when  N/p  =  20.    Johansen's  test 
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performed  well  in  all  conditions,  but  performed  better  in  the 
positive  conditions.  Its  performance  in  the  negative  conditions  was 
better  when  for  d  =  JTT5  than  for  d  =  3.0.  In  terms  of  number  of 
rs  outside  the  criterion  interval,  the  second  order  test  performed 
more  poorly  than  Johansen's  test.  However,  its  performance  was  not 
bad  and  may  be  preferred  by  researchers  who  do  not  want  to  risk  r 
>  q.  A  practical  problem  for  such  researchers  will  be  the 
difficulty  in  programming  the  second  order  test. 
Summary 

Six  factors,  the  ratio  of  N  to  p,  the  magnitude  of  d  in  D,  the 
sample-size-covariance  relationship,  the  value  of  p,  the  sample  size 
ratio,  and  the  form  of  D,  affected  the  performance  of  the  four  tests 
studied.  However,  the  magnitude  and  the  direction  of  the  effect 
were  not  the  same  across  tests. 

In  general,  as  N/p  increased,  all  of  the  tests  except  the 
Pillai-Bartlett  trace  become  more  robust.  The  effect  of  the  other 
factors  on  Johansen's  test  and  James'  tests  seems  to  decrease  as  N/p 
increased:  the  number  of  fs  within  the  criterion  interval  increased 
as  N/p  increased.  Even  though  James'  first  order  test  performed 
better  as  N/p  increased  to  20,  it  had  more  rs  outside  the  criterion 
interval  than  the  second  order  test  and  Johansen's  test. 

As  d  increased,  with  the  exception  of  the  second  order  test 
when  N/p  -  20,  the  performance  of  all  the  tests  declined.   For  N/p 
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=  20,  James'  second  order  test  performed  as  well  or  better  as  d 
increased . 

The  rs  tend  to  be  larger  in  the  negative  condition  than  in  the 
positive  condition.  This  effect  lead  to  a  better  performance  in  the 
positive  condition  for  the  second  order  test  and  a  poorer 
performance  for  Johansen's  test  and  the  first  order  test.  For  the 
Pillai-Bartlett  trace,  is  are  much  smaller  than  a  in  the  positive 
condition  and  much  larger  in  the  negative  condition. 

When  the  sample  size  ratio  becomes  more  extreme,  the 
performance  of  the  tests,  except  the  second  order  test,  declines. 
The  extreme  sample  size  ratio  lead  to  a  moderately  better 
performance  on  the  second  order  test  when  N/p  was  less  than  20. 

The  change  in  p  had  a  strong  impact  on  the  performance  of  the 
second  order  test.  The  test  was  more  robust  when  p  =  3  than  when 
p  =  6.   This  factor  did  not  impact  on  the  other  tests. 

The  form  of  D  affected  only  the  Pillai-Bartlett  trace.  When 
D  is  in  the  form  of  diag(d2  or  1/d2)  the  performance  of  the  test 
improved. 

Tables  38,  39,  and  40  were  constructed  using  the  rule  of  thumb 
that  a  test  is  robust  for  a  particular  condition  if  three  or  fewer 
ts  are  outside  the  criterion  interval.  These  serve  to  summarize 
the  major  effects  of  the  factors  on  the  four  tests.   Similar  results 
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Table  38 


Number  of  fs  Outside  the  Criterion  Interval  of  the  Four  Multivariate 
Tests  for  N/p  =  10  and  a   =  .05 


d    p   Positive/   Pillai-    Johansen's   James'      James' 
Negative    Bartlett      test    1st  order   2nd  order 
trace  test        test 


073   3      P 

N 

6      P 

N 
3    3      P 

N 
6      P 

N 


8 

7 

12 

7 

2* 

10 

12 

2* 

7 

10 

12 

11 

3' 

10 

12 

10 

9 

5 

12 

6 

12 

12 

12 

3* 

12 

9 

12 

12 

11 

11 

12 

10 

Note.   A  *  indicates  the  test  is  relatively  robust.   The  maximum 
number  of  ts  in  each  cell  is  12. 


are  reported  in  Tables  41  to  43  for  a  =  .01  and  in  Tables  44  to  46 
for  a  =  .10.  These  substantiate  the  claim  that  major  patterns  of 
effects  are  similar  for  all  three  a  levels.  The  most  notable 
exception  to  this  claim  in  for  the  second  order  test  when  a  -  .01. 
It  was  more  robust  than  Johansen's  test  for  both  N/p  =  15  and  20. 
This  is  a  floor  effect:  the  second  order  test  tended  to  have  t  <  a. 
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Table  39 


Number  of  £s  Outside  the  Criterion  Interval  of  the  Four  Multivariate 
for  N/p  =  15  and  a   =  .05 


d    p   Positive/   Pillai-    Johansen's   James'      James' 
Negative    Bartlett      test    1st  order   2nd  order 
trace  test         test 


a. 5  3     P 

N 
6      P 

N 
3   3      P 

N 
6      P 

N 


6 

0' 

6 

7 

4 

12 

6 

3* 

9 

5 

6 

10 

8 

2* 

9 

12 

8 

11 

11 

1* 

6 

12 

10 

12 

4 
0* 
8 
3* 
2* 
0* 
12 
9 


Note.   A  *  indicates  the  test  is  relatively  robust.   The  maximum 
number  of  rs  in  each  cell  is  12. 
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Table  40 


Number  of  fs  Outside  the  Criterion  Interval  of  the  Four  Multivariate 
Tests  for  N/p  -  20  and  a   -    .05 


d    p    Positive/    Pillai-     Johansen's    James'       James' 
Negative    Bartlett      test    1st  order   2nd  order 
trace  test         test 


AT75   3 


p 

6 

0* 

2* 

2* 

N 

4 

0' 

7 

1* 

P 

5 

1* 

4 

6 

N 

4 

2* 

5 

4 

P 

9 

0* 

3' 

0" 

N 

12 

3* 

11 

1* 

P 

10 

1* 

4 

5 

N 

11 

3' 

8 

3* 

Note.   A  *  indicates  the  test  is  relatively  robust.   The  maximum 
number  of  rs  in  each  cell  is  12. 
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Table  41 


Number  of  £s  Outside  the  Criterion  Interval  of  the  Four  Multivariate 
Tests  for  N/p  =  10  and  a   =    .01 


d    p    Positive/    Pillai-     Johansen's    James'       James' 
Negative    Bartlett      test    1st  order   2nd  order 
trace  test        test 


OT5   3      P 


5  4         12  2' 


N  5  4  12  0* 

P  6  7  12  5 

N  10  11  12  4 

P  5  6  12  4 

N  10  7  12  9 

P  4  8  12  8 

N  11  11  12  5 


Note.   A  *  indicates  the  test  is  relatively  robust.   The  maximum 
number  of  rs  in  each  cell  is  12. 
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Table  42 


Number  of  fs  Outside  the  Criterion  Interval  of  the  Four  Multivariate 
Tests  for  N/p  -  15  and  a   =  .01 


d    p    Positive/    Pillai-     Johansen's    James'       James' 
Negative     Bartlett       test     1st  order    2nd  order 
trace  test        test 


173   3 


4 

5 

3* 
10 

5 

8 

3* 
12 


1" 
1" 
1* 
7 
4 
4 
2* 
5 


6 

12 

8 

9 

9 

10 

8 

11 


Note.   A  *  indicates  the  test  is  relatively  robust.   The  maximum 
number  of  rs  in  each  cell  is  12. 
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Table  43 


Number  of  £s  Outside  the  Criterion  Interval  of  the  Four  Multivariate 
Testsfor  N/p  =  20  and  a  =  .01 


d    p    Positive/    Pillai-     Johansen's    James'       James' 
Negative    Bartlett      test    1st  order    2nd  order 
trace  test        test 


JTTs 

3 

P 

4 

0' 

5 

1* 

N 

7 

0' 

9 

1" 

6 

P 

2* 

0* 

4 

1* 

N 

10 

4 

6 

2* 

3 

3 

P 

3* 

2' 

5 

3' 

N 

10 

5 

6 

3* 

6 

P 

4 

1* 

4 

2* 

N 

11 

4 

8 

2* 

Note. 

A  * 

indicates 

the  test 

is 

relatively 

robust . 

The 

maximui 

number  of  rs  in  each  cell  is  12 
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Table  44 


Number  of  rs   Outside  the  Criterion  Interval  of  the  Four  Multivariate 
Tests  for  N/p  =  10  and  a  =  . 10 


d    p    Positive/    Pillai-     Johansen's    James'       James' 
Negative     Bartlett       test     1st  order    2nd  order 
trace  test         test 


<T73   3 


3    3 


10 

4 

6 

6 

11 

12 

12 

12 


8 
11 
12 
12 

7 
12 
12 
10 


12 
12 
12 
12 
11 
12 
12 
12 


5 

4 
12 
11 

5 

3* 
12 
12 


Note.   A  *  indicates  the  test  is  relatively  robust.   The  maximum 
number  of  rs  in  each  cell  is  12. 
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Table  45 


Number  of  £s  Outside  the  Criterion  Interval  of  the  Four  Multivariate 
Tests  for  N/p  =  15  and  a   =  . 10 


d    p    Positive/    Pillai-     Johansen's    James'       James' 
Negative    Bartlett      test    1st  order   2nd  order 
trace  test         test 


OT5   3 


p 

7 

N 

6 

P 

8 

N 

5 

P 

9 

N 

12 

P 

11 

N 

11 

2' 

6 

6 

12 

u 

9 

6 

9 

3' 

7 

8 

11 

1" 

5 

8 

12 

5 

1" 

9 

9 

2* 

2* 
12* 
10 


Note .   A  *  indicates  the  test  is  relatively  robust.   The  maximum 
number  of  ts  in  each  cell  is  12. 
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Table  46 


Number  of  fs  Outside  the  Criterion  Interval  of  the  Four  Multivariate 
Tests  for  N/p  =  20  and  a  =  . 10 


d    p    Positive/    Pillai-     Johansen's    James'       James' 
Negative    Bartlett      test    1st  order   2nd  order 
trace  test        test 


/T75   3 


6 

7 

7 

5 

11 

12 

10 

10 


2* 
0* 
3" 
3* 
0* 
4 
2* 
2* 


3* 
5 
4 

7 
3* 

7 
6 

10 


3* 
0* 
4 

6 

0" 
2* 
4 

5 


Note .   A  *  indicates  the  test  is  relatively  robust.   The  maximum 
number  of  ts  in  each  cell  is  12. 


CHAPTER  5 
CONCLUSIONS 

The  results  of  the  study  provide  guidelines  for  conducting 
multivariate  tests  when  variance-covariance  matrices  in  the  k 
populations  from  which  the  samples  are  drawn  are  heterogeneous.  The 
results  obtained  can  be  applied  to  experiments  which  have 
experimental  conditions  similar  to  the  360  simulated  experiments 
conducted  in  this  study.  The  general  conclusion  is  that  no  test 
among  the  four  performs  uniformly  well;  that  is,  the  performance  of 
the  tests  depends  on  the  experimental  conditions.  James'  first 
order  test  performed  more  poorly  than  the  other  tests  in  terms  of 
controlling  Type  I  error  rate  under  all  360  experiments  in  this 
study.  Therefore  it  is  not  recommended  for  use.  The  Pillai- 
Bartlett  trace  performed  poorly  except  when  n,  -  r^  =  n3  and  d  = 
7175.  When  the  test  is  not  robust,  it  tends  to  be  conservative  when 
the  smaller  samples  are  from  populations  with  smaller  variance- 
covariance  matrices  and  liberal  when  the  smaller  samples  are  from 
populations  with  larger  variance-covariance  matrix.  Johansen's  test 
and  James'  second  order  test  perform  better  than  the  other  two 
tests.  When  Johansen's  test  is  not  robust,  it  tends  to  be  liberal. 
When  James  second  order  test  is  not  robust,  it  tends  to  be 
conservative . 
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Six  factors  affected  the  performance  of  the  tests,  namely, 
the  ratio  of  total  sample  size  to  the  number  of  variable  (N/p) ,  the 
magnitude  of  dispersion  of  the  covariance  matrices  (d) ,  sample-size- 
covariance -matrix  relationship  (negative  or  positive),  sample  size 
ratio  (n^iuing),  number  of  variables  (p)  ,  and  form  of  the  covariance 
matrices  (D) .  Among  these  six  factors,  the  ratio  of  total  sample 
size  to  number  of  variables  (N/p)  had  the  dominant  effect  on  the 
performance  of  the  tests.  As  N/p  increased  from  10  to  20,  the 
performance  of  the  tests,  except  the  Pillai -Bartlett  trace, 
improved. 

When  d  increased  from  71.5  to  3.0,  the  performance  of  the 
tests  typically  declined.  The  degree  of  decline  increased  as  N/p 
increased.  There  is  an  exception  for  James'  second  order  test. 
When  N/p  -  20,  the  test  performs  better  when  d  is  large. 

The  sample-size-covariance-matrix  relationship  also  had  a 
relatively  strong  impact  on  the  performance  on  the  tests.  The 
performance  of  the  Pillai-Bartlett  trace  follows  the  pattern  of 
Hotelling's  T2:  the  test  is  conservative  under  the  positive 
condition  and  liberal  under  the  negative  condition.  Johansen's  test 
and  James'  first  order  test  follow  the  same  pattern  as  Yao ' s  and 
James'  tests  when  p  =  2.    They  perform  better  in  the  positive 
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condition  than  in  the  negative  condition.  James  second  order  test 
performs  better  in  negative  condition  than  in  positive  condition. 
When  N/p  increased  to  20,  the  effect  of  this  factor  on  the  tests 
decreased. 

When  the  sample  size  ratio  was  more  extreme,  the  performance 
of  the  tests  except  James'  second  order  test  declined.  Extreme 
sample  size  ratios  lead  to  a  moderately  better  performance  on  the 
second  order  test  when  N/p  was  less  than  20. 

The  change  in  p  has  a  strong  impact  on  the  performance  of  the 
second  order  test.  The  test  was  more  robust  when  p  =  3  than  when 
p  =  6.   This  factor  did  not  impact  the  other  tests. 

The  form  of  D  only  effected  the  Pillai -Bartlett  trace.  When 
D  is  in  the  form  of  diag(d2  or  1/d2)  the  performance  of  the  test 
improved. 

The  generalizability  of  the  results  is  limited  by  the  range 
of  variation  in  the  factors  and  the  use  of  multivariate  normal  data. 
Keeping  these  limitations  in  mind  the  following  conclusions  and 
recommendations  can  be  set  forth. 

Experiments  with  Equal  Sample  Sizes 

Conclusion  1.  When  the  heterogeneity  among  the  three 
population  covariance  matrices  is  small,  such  as  d  <  71 . 5 ,  Pillai- 
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Bartlett  trace,  the  traditional  multivariate  test,  can  be  used 
safely.  Johansen's  test  also  can  be  used;  its  performance  is  as 
good  as  the  Pillai-Bartlett  trace  when  N/p  >  15  and  almost  as  good 
when  N/p  =  10. 

Conclusion  2.  When  d  >  3,  Johansen's  test  should  be  used  to 
obtain  an  accurate  type  I  error  rate. 

Recommendation  1.  When  sample  sizes  are  equal  Johansen's  test 
should  replace  the  multivariate  criteria  based  on  the  homogeneity 
assumption.  Researchers  should  attempt  to  obtain  an  N/p  ratio  of 
at  least  15. 

Experiments  with  Unequal  Sample  Sizes 

Conclusion  3.  When  N/p  =  10,  d  <  71.5,  and  sample-size- 
covariance -matrix  (SSCM)  relationship  is  negative,  the  Pillai- 
Bartlett  trace  can  be  used  for  both  p  -  3  or  p  =  6 . 

Conclusion  4.  With  N/p  -  10 ,  p  -  3 ,  and  the  SSCM  relationship 
is  negative,  James'  second  order  test  performs  well. 
Recommendation  2.  When  N/p  -  10  the  best  choice  is  James'  second 
order  test.  It  is  typically  conservative,  but  not  extremely  so. 
The  other  tests  can  be  extremely  liberal.  It  is,  however,  best  to 
avoid  N/p  ratios  this  small  because  programming  the  second  order 
test  is  extremely  complicated. 
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Conclusion  5.  When  N/p  -  15  and  the  SSCM  relationship  is 
positive,  Johansen's  test  works  well. 

Conclusion  6.  When  N/p  -  15,  d  <  71 .  5 ,  and  the  SSCM 
relationship  is  negative,  James'  second  order  test  works  well.  It 
also  performs  well  when  d  <  3  and  p  <  3.  When  d  >  3  and  p  >  6,  no 
test  performs  well. 

Conclusion  7  .  When  N/p  =  20,  Johansen's  test  works  well.  The 
second  order  test  words  well  provided  p  <  3. 

Conclusion  8.  When  N/p  -  20,  the  SSCM  relationship  is 
negative,  and  p  <  6,   the  second  order  test  works  well. 

Recommendation  3.  When  N/p  >  15  both  Johansen's  test  and 
James  second  order  test  can  be  used  because  they  each  have  r 
reasonably  near  a.  The  choice  can  be  made  based  on  the  following 
considerations:  James'  second  order  test  can  be  somewhat 
conservative;  Johansen's  test  can  be  somewhat  liberal.  Johansen's 
test  is  substantially  easier  to  program. 


APPENDIX  A 

RESULTS  OF  THE  SIMULATED  EXPERIMENTS 

WHEN  n,  =  tl,  =  n3 ,  a   =  .01,  AND  a  -  .  10 


Table  47 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
n]  -  ru  -  n3.  D  =  d2I .  d  =  ./T3.  and  a   =    .01 


N/p      Pillai-      Johansen's     James'       James' 
Bartlett      test      1st  order   2nd  order 
trace  test        test 


I:I:D 


3 

10 

.0070 

.0100 

.0180 

.0050 

3 

15 

.0095 

.0085 

.0120 

.0060 

3 

20 

.0065 

.0075 

.0075 

.0070 

6 

10 

.0090 

.0115 

.0190 

.0030 

6 

15 

.0075 

.0105 

.0145 

.0065 

6 

20 

.0085 

.0095 

.0115 

.0070 

I:D:D 


3 

10 

.0055 

.0135 

3 

15 

.0095 

.0115 

3 

20 

.0080 

.0065 

6 

10 

.0085 

.0110 

6 

15 

.0085 

.0095 

6 

20 

.0075 

.0150 

,0190  . 0060 

.0150  .0080 

.0105  .0060 

.0160  .0035 


.0120       .0065 
.0145       .0065 


Note .   The  underlined  fs  are  outside  the  interval  a   ±  27q(1-q)/2000 . 
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Table  48 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
0,  -  tu  -  n3.  D  -  diagCd2  d2  1)    or  D  -  diae(d2  d?  d2  1  1  1) 
d  =  Jl . 5 .  and  a   =  .  01 


p     N/p     Pillai-      Johansen's     James'       James' 
Bartlett      test      1st  order   2nd  order 


trace 

test 

test 

I:I:D 

3 

10 

.0085 

.0155 

.0215 

.0095 

3 

15 

.0075 

.0090 

.0125 

.0075 

3 

20 

.0095 

.0100 

.0140 

.0075 

6 

10 

.0115 

.0140 

.0255 

.0040 

6 

15 

.0110 

.0095 

.0120 

.0045 

6 

20 

.0105 

.0110 

.0145 

.0085 

I:D:D 


3 

10 

.0075 

.0100 

.0180 

.0045 

3 

15 

.0085 

.0090 

.0140 

.0065 

3 

20 

.0125 

.0110 

.0135 

.0090 

6 

10 

.0100 

.0090 

.0160 

.0045 

6 

15 

.0085 

.0100 

.0135 

.0055 

6 

20 

.0075 

.0070 

.0100 

.0060 

Note.   The  underlined  fs  are  outside  the  interval  a  ±  27a(l-a)/2000 
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Table  49 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 

n     -   ru  =   n       D   -   diapCd2   d2    1/d2)    or 

D   =   diag(d?  d2   d2    1/d2    1/d2    1/d2).    d   =   TO.    and   a   =    .01 


N/p     Pillai-     Johansen's    James'      James' 
Bartlett      test      1st  order   2nd  order 
trace  test        test 


I :  I :  D 


I;D:D 


3 

10 

.0070 

.0115 

.0205 

.0080 

3 

15 

.0110 

.0140 

.0185 

.0115 

3 

20 

.0060 

.0065 

.0100 

.0050 

6 

10 

.0080 

.0110 

.0165 

.0050 

6 

15 

.0095 

.0125 

.0145 

.0070 

6 

20 

.0110 

.0110 

.0140 

.0075 

3 

10 

.0100 

.0120 

.0205 

.0075 

3 

15 

.0095 

.0095 

.0135 

.0080 

3 

20 

.0060 

.0105 

.0120 

.0095 

6 

10 

.0090 

.0180 

.0265 

.0055 

6 

15 

.0095 

.0125 

.0160 

.0070 

6 

20 

.0090 

.0105 

.0125 

.0080 

Note .   The  underlined  fs  are  outside  the  interval  a  ±  2ja(l-a) /2000 . 
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Table  50 

Actual  Tvpe 

I  Error  Rates  for  Four 

Multivariate 

Tests : 

Bi  ■  "  "a  "  Vv 

D  =  d2I,  d  - 

=  3 .  and  a 

=  .01 

P     N/p 

Pillai- 

Johansen' 

s     James' 

James ' 

Bartlett 

test 

1st  order 

2nd  order 

trace 

test 

test 

I:I:D 

3     10 

.0205 

.0090 

.0215 

.0060 

3     15 

.0270 

.0110 

.0160 

.0090 

3     20 

.0265 

.0125 

.0135 

.0115 

6     10 

.0295 

.0165 

.0295 

.0060 

6     15 

.0210 

.0115 

.0140 

.0075 

6     20 

.0205 

.0090 

.0115 

.0055 

I:D:D 

3     10 

.0100 

.0135 

.0200 

.0075 

3     15 

.0125 

.0085 

.0145 

.0065 

3     20 

.0155 

.0110 

.0150 

.0080 

6     10 

.0175 

.0175 

.0207 

.0035 

6     15 

.0160 

.0105 

.0160 

.0060 

6     20 

.0175 

.0105 

.0130 

.0055 

Note.   The  underlined  fs  are  outside  the  interval  a  ±  2,/a(l-a)/2000 . 


Table  51 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
n,  -  ru  -  ns.  D  -  diag(d2  d2  1)  or  D  -  diagfd2  d2  d2  1  1  1) 
d-3,  and  a  =  . 01 
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N/p 


Pillai-     Johansen's     James'       James' 
Bartlett     test       1st  order   2nd  order 
trace  test        test 


I:I:D 


3 

10 

.0130 

.0140 

.0250 

.0105 

3 

15 

.0175 

.0090 

.0105 

.0055 

3 

20 

.0195 

.0095 

.0130 

.0080 

6 

10 

.0190 

.0120 

.0205 

.0040 

6 

15 

.0190 

.0120 

.0155 

.0075 

6 

20 

.0200 

.0125 

.0150 

.0110 

I:I:D 


3 

10 

.0095 

.0105 

.0215 

.0080 

3 

15 

.0125 

.0125 

.0195 

.0085 

3 

20 

.0115 

.0105 

.0115 

.0090 

6 

10 

.0075 

.0135 

.0195 

.0045 

6 

15 

.0135 

.0140 

.0165 

.0090 

6 

20 

.0105 

.0055 

.0070 

.0040 

Note.   The  underlined  fs  are  outside  the  interval  a  ±  27q(1-q)/2000 
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Table  52 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 

n,   -  tu  -  Tig.    D  -   diafrfd2  d2   1/d2)    or 

D  =   diag(d?  d2   d2   1/d2   1/d2   1/d2).    d  -   3.    and  a  =    .01 


3 

10 

3 

15 

3 

20 

6 

10 

6 

15 

6 

20 

N/p     Pillai-      Johansen's     James'       James' 
Bartlett      test      1st  order   2nd  order 
trace  test        test 


I:I:D 


.0230 
.0190 
.0210 
.0205 
.0275 
.0225 


.0150 

.0235 

.0075 

.0105 

.0150 

.0080 

.0055 

.0075 

.0040 

.0170 

.0250 

.0085 

.0140 

.0175 

.0080 

.0065 

.0105 

.0055 

I:I:D 


3 

10 

.0200 

.0130 

.0240 

.0085 

3 

15 

.0190 

.0110 

.0180 

.0090 

3 

20 

.0175 

.0065 

.0100 

.0050 

6 

10 

.0215 

.0130 

.0210 

.0050 

6 

15 

.0200 

.0140 

.0170 

.0090 

6 

20 

.0225 

.0075 

.0100 

.0050 

Note. 

The 

unde 

rlined  fs 

are  outside 

the 

interval  a   ±  2ya(l-Q)/2000 

Ill 


Table  53 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Si  =  a,  -  n3,  p  -  d2I.  d  =  7173.  and  a   =  .  10 


p     N/p      Pillai-    Johansen's     James'       James' 
Bartlett     test       1st  order   2nd  order 
trace  test         test 


I:I:D 

3 

10 

.0950 

.0101 

.1170 

.0725 

3 

15 

.1040 

.1095 

.1125 

.0955 

3 

20 

.0955 

.1030 

.1085 

.0945 

6 

10 

.0875 

.0150 

.1240 

.0615 

6 

15 

.0880 

.1010 

.1075 

.0770 

6 

20 

.0965 

.0935 

.0980 

.0805 

I:D:D 


3 

10 

.0955 

.1140 

.1285 

.0785 

3 

15 

.1010 

.1075 

.1170 

.0935 

3 

20 

.1010 

.1080 

.1140 

.0995 

6 

10 

.0940 

.1160 

.1305 

.0665 

6 

15 

.0910 

.0960 

.1025 

.0740 

6 

20 

.1070 

.1125 

.1190 

.0985 

Note .   The  underlined  fs  are  outside  the  interval  a  ±  27a(l-a)/2000 
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Table  54 


Actual  Type 

T 

F 

rror  Rates  for 

Four 

Multivariate 

Tests 

ni  = 

ru  =   n3 

r 

• 

.  diap(d2 

d2  1) 

or 

D 

-  diag 

d2  d2 

d2 

1  1 

1), 

d  = 

JT75.    and 

Q 

=  .10 

N/p    Pillai-    Johansen's 
Bartlett     test 
trace 


James'       James' 
1st  order   2nd  order 
test         test 


I:I:D 


3 

10 

.0900 

.1145 

.1320 

.0895 

3 

15 

.0955 

.1040 

.1110 

.0920 

3 

20 

.1115 

.1125 

.1160 

.1050 

6 

10 

.1065 

.1275 

.1425 

.0810 

6 

15 

.1125 

.1125 

.1235 

.0930 

6 

20 

.1070 

.1120 

.1195 

.0935 

I:D:D 


3 

10 

.0955 

.1100 

.1230 

.0850 

3 

15 

.1060 

.1085 

.1170 

.0980 

3 

20 

.0870 

.0900 

.0935 

.0865 

6 

10 

.0965 

.1155 

.1355 

.0705 

6 

15 

.1030 

.1020 

.1105 

.0765 

6 

20 

.1070 

.1085 

.1125 

.0945 

Note.   The  underlined  fs  are  outside  the  interval  a  ±  2,/a(l-a)/2000 . 
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Table  55 

Actual  Tvpe  I  Error  Rates  for  Four  Multivariate  Tests: 

B,   -  a,  -  Tu.    D  -   diag(d2   d2   1/d2)    or 

D  -   diagfd2"  d2   d2   1/d2   1/d2   1/d2).    d  -  7175.    and  o  -    .  10 


N/p  Pillai-  Johansen's  James'  James' 

Bartlett  test  1st   order        2nd  order 

trace  test  test 


I:I:D 


I:D:D 


3 

10 

.0970 

.1185 

.1310 

.0885 

3 

15 

.0975 

.1015 

.1090 

.0900 

3 

20 

.0925 

.0970 

.1035 

.0875 

6 

10 

.0905 

.1155 

.1275 

.0660 

6 

15 

.0915 

.0965 

.1075 

.0825 

6 

20 

.1070 

.1120 

.1185 

.1000 

3 

10 

.0955 

.1055 

.1260 

.0810 

3 

15 

.1005 

.1005 

.1085 

.0905 

3 

20 

.0935 

.0950 

.0995 

.0875 

6 

10 

.0980 

.1235 

.1385 

.0775 

6 

15 

.0940 

.0925 

.1025 

.0740 

6 

20 

.1070 

.1120 

.1145 

.0995 

Note.   The  underlined  fs  are  outside  the  interval  a   ±    2ja{ l-a)/2000 . 
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Table  56 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
n,  =  ru  =  ru,  D  -  d2I .  d  -   3.  and  a  =    .  10 


N/p      Pillai-    Johansen's     James'      James' 

Bartlett     test       1st  order   2nd  order 
trace  test        test 


I:I:D 

3 

10 

.1520 

.1250 

.1445 

.0920 

3 

15 

.1415 

.1045 

.1135 

.0900 

3 

20 

.1470 

.1055 

.1090 

.0975 

6 

10 

.1465 

.1180 

.1340 

.0715 

6 

15 

.1425 

.1040 

.1110 

.0810 

6 

20 

.1355 

.0915 

.0975 

.0790 

I:I:D 

10 

3 

.1080 

.1170 

.1465 

.0840 

15 

3 

.1105 

.1105 

.1210 

.0900 

20 

3 

.1060 

.1030 

.1090 

.0880 

10 

3 

.1140 

.1340 

.1585 

.0765 

15 

3 

.1185 

.1160 

.1270 

.0805 

20 

3 

.1335 

.1130 

.1225 

.0965 

Note .   The  underlined  fs  are  outside  the  interval  a  ±  2ya(l-a)/2000 . 
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Table  57 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
ni  =  n^  =  n3.  D  -  diagfd2  d2  1)  or  D  -  diae(d2  d2  d2  1  1  1) 
d  =  3 .  and  a   =    . 10 


N/p      Pillai-    Johansen's     James'       James' 

Bartlett     test       1st  order   2nd  order 
trace  test       test 


I:I:D 


3 

10 

.1265 

.1125 

.1280 

.0880 

3 

15 

.1295 

.0880 

.0980 

.0715 

3 

20 

.1205 

.0980 

.1035 

.0910 

6 

10 

.1305 

.1305 

.1455 

.0765 

6 

15 

.1260 

.1030 

.1095 

.0770 

6 

20 

.1270 

.1045 

.1095 

.0930 

I:D:D 


3 

10 

.1085 

.1210 

.1400 

.0840 

3 

15 

.1110 

.1145 

.1215 

.0975 

3 

20 

.1115 

.0995 

.1070 

.0915 

6 

10 

.0940 

.1220 

.1395 

.0650 

6 

15 

.0955 

.1030 

.1125 

.0770 

6 

20 

.1045 

.0935 

.0985 

.0785 

Note .   The  underlined  fs  are  outside  the  interval  a  ±  2ya(l-a)/2000. 
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Table  5f 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests 

n    ~  ru  -  n       D  -   diag(d2  d2   1/d2)    or 

D  -   diaeCd2^  d2  d2   1/d2   1/d2   1/d2).    d  -   3.    and  o  -    .10 


p     N/p     Pillai-      Johansen's     James'       James' 
Bartlett      test      1st  order    2nd  order 
trace  test        test 


I:I:D 


3 

10 

.1300 

.1245 

.1380 

.0905 

3 

15 

.1325 

.1045 

.1180 

.0940 

3 

20 

.1275 

.0960 

.1025 

.0860 

6 

10 

.1235 

.1215 

.1355 

.0780 

6 

15 

.1330 

.1245 

.1365 

.1000 

6 

20 

.1280 

.0990 

.1055 

.0865 

I:D:D 


3 

10 

.1235 

.1220 

.1350 

.0865 

3 

15 

.1195 

.1065 

.1120 

.0890 

3 

20 

.1195 

.1065 

.1120 

.0975 

6 

10 

.1290 

.1305 

.1435 

.0800 

6 

15 

.1265 

.1105 

.1180 

.0875 

6 

20 

.1285 

.1015 

.1040 

.0895 

Note  . 

The 

unde 

rlined  fs 

are 

outside 

the 

interval  a   ±  27q(1-q)/2000 

APPENDIX  B 

RESULTS  OF  THE  SIMULATED  EXPERIMENTS 

WHEN  SAMPLE  SIZES  ARE  UNEQUAL  AND  a   =  .01 


Table  59 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition.  N/p  =10.  p  -  3.  d  =  71 ■ 5 .  and  a  -  .01 


"1   "2   n3 


Pillai-     Johansen's     James'        James' 
Bartlett     test       1st  order    2nd  order 
trace  test         test 


D  - 

d?I 

9 

9 

12 

.0065 

.0105 

.0195 

.0065 

7 

7 

16 

.0040 

.0135 

.0275 

.0080 

8 

11 

11 

.0070 

.0120 

.0190 

.0075 

6 

12 

12 

.0050 

.0190 

.0315 

.0085 

D  - 

diag 

(d2 

d2  1) 

9 

9 

12 

.0075 

.0100 

.0180 

.0065 

7 

7 

16 

.0030 

.0140 

.0265 

.0075 

8 

11 

11 

.0040 

.0080 

.0165 

.0040 

6 

12 

12 

.0065 

.0205 

.0335 

.0085 

D  = 

diag 

(d2 

d2  1/d2) 

9 

9 

12 

.0050 

.0110 

.0170 

.0055 

7 

7 

16 

.0060 

.0200 

.0330 

.0090 

8 

11 

11 

.0065 

.0110 

.0185 

.0070 

6 

12 

12 

.0110 

.0160 

.0250 

.0090 

Note.   When  n1  =  n^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n  *  n2  -  n3>  the  form  of  contamination  is  I:D:D.   The  underlined 
fs  are  outside  the  interval  of  a  i  27a(l-»)/2000 . 
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Table  60 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition.  N/p  =  10,  P  -  3,  d  =  71.5.  and  a  =    .01 
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N/p      n,      ix,      n3 


Pillai-  Johansen's  James'  James' 

Jartlett  test  1st   order        2nd   order 

trace  test  test 


D   -    dzI 

9  9  12 

7  7  16 

8  11  11 

6  12  12 
D   -   diag(d2  d2    1) 

9  9  12 

7  7  16 

8  11  11 

6  12  12 
D  =   diag(d2  d2   1/d2) 

9  9  12           .0130 

7  7  16           .0110 

8  11  11     .0250 
6  12  12     .0330 


.0095 

.0115 

.0185 

.0185 

.0130 

.0155 

.0080 

.0200 

.0085 

.0120 

.0010 

.0110 

.0060 

.0115 

.0045 

.0115 

0090 
,0185 
.0135 
.0125 


.0195 


,0405 


.0250 


0375 


0175 


,0230 


,0225 


0200 


.0195 


0295 


.0210 


.0220 


.0065 
.0095 
.0105 
.0120 

.0080 
.0065 
.0065 

.0075 

.0050 
.0090 
.0100 
.0065 


Note.   When  n,  «  r^  *  n3,  the  form  of  contamination  is  I:I:D;  when 
n1  *  ru  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  a  i  2,/a(l-a)/2000 . 
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Table  61 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  10 .  p  -  6  .  d  =  ./l  .  5  .  and  a   -    .01 


N/p  n.   n_  n3     Pillai-     Johansen's    James' 


James ' 


Bartlett     test       1st  order   2nd  order 
trace  test        test 


0040 

.0115 
.0210 

.0205 
.0310 

.0040 

.0035 

.0070 

.0060 

.0105 

.0170 

.0035 

.0045 

.0110 

.0230 

.0055 

D  -  d2I 

18  18  24 

15  15  30 

16  22  22 
12  24  24 

D  =  diag(d2  d2  d2  1  1  1) 

9  9  12  .0080 

7  7  16  .0140 

8  11  11  .0125 

6  12  12  .0145 
D  =  diag(d2  d2  d2  1/d2  1/d2  1/d2) 

9  9  12  .0055  .0060        .0165       -0030 

7  7  16  .0055  .0210        .0340       .0070 

8  11  11  .0075  .0155        .0260       .0085 
6  12  12  .0115       .0195        .0335       .0130 


.0105 

.0235 

.0070 

.0220 

.0390 

.0100 

.0150 

.0225 

.0115 

.0285 

.0425 

.0145 

Note.   When  n,  =  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *   nj  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
fs  are  outside  the  interval  of  a  ±  27a( l-a)/2000 . 
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Table  62 


Negative 

Con 

dition,  N/p  = 

10. 

P 

-  6.  d  - 

JX 

5  and  a  = 

.01 

N/p  n. 

n^ 

n3 

Pillai- 
Batlett 

J 

Dhansen' s 
test 

James ' 
1st  order 

James ' 
2nd  order 

trace 

test 

test 

D  -  d2I 

18 

18 

24 

.0105 

.0135 

.0200 

.0035 

15 

15 

30 

.0180 

.0185 

.0305 

.0070 

16 

22 

22 

.0135 

.0205 

.0285 

.0075 

12 

24 

24 

.0160 

.0265 

.0415 

.0095 

D  =  diag 

(d2 

d2  d2 

1  1  1) 

9 

9 

12 

.0185 

.0155 

.0295 

.0085 

7 

7 

16 

.0505 

.0255 

.0490 

.0120 

8 

11 

11 

.0395 

.0185 

.0300 

.0110 

6 

12 

12 

.0795 

.0405 

.0630 

.0235 

D  -  diag 

(d2 

d2  d2 

1/d2  1/d2 

1/d2) 

9 

9 

12 

.0160 

.0185 

.0335 

.0115 

7 

7 

16 

.0360 

.0295 

.0500 

.0165 

8 

11 

11 

.0410 

.0165 

.0250 

.0090 

6 

12 

12 

.0740 

.0315 

.0435 

.0190 

Note.   When  n.  =  n^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *   r^  -  n3,  the  form  of  contamination  is  I : D : D^  The  underlined 
fs  are  outside  the  interval  of  a   ±  27q(1-q)/2000 . 


Table  63 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  10.  p  -  3,  d  =  3,  and  a  =  .01 
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N/p  n,   rt,  n3 


Pillai- 
lartlett 
trace 


.0070 

.0100 

.0000 

.0150 

.0055 

.0170 

.0300 

.0135 

D  =  d2I 

9  9  12 

7  7  16 

8  11  11 
6  12  12 

D  =   diag(d2  d2   1) 

18  18  24 

15  15  30 

16  22  22 
12  24  24 

D  -  diag(d2  d2  1/d2) 

18  18  24     .0040 

15  15  30     .0075 

16  22  22     .0095 
12  24  24     .0095 


Johansen's     James'       James' 
test     1st  order   2nd  order 
test       test 


.0045 

.0135 

.0075 

.0185 

.0045 

.0130 

.0070 

.0195 

.0105 
.0145 
.0135 
.0205 


.0190 


.0195 


.0300 


.0240 


0220 


.0275 


0215 


.0285 


.0175 


0245 


.0210 


0360 


.0070 
.0090 
.0105 
.0085 

.0040 
.0065 
.0050 
.0065 

.0015 
.0060 
.0045 
.0070 


Note.   When  n,  =  v^  *   n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *  tv,  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
fs  are  outside  the  interval  of  a  i  2ja(l-a)/200Q . 
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Table  64 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =  10.  p  =  3.  d-3.  and  a 


01 


N/p  a.   rij  n3    Pillai-     Johansen's    James'      James' 
Bartlett      test       1st  order   2nd  order 


trace 

test 

test 

D  - 

■  d2I 

9 

9 

12 

.0245 

.0215 

.0385 

.0130 

7 

7 

16 

.0970 

.0360 

.0620 

.0180 

8 

11 

11 

.0610 

.0315 

.0490 

.0205 

6 

12 

12 

.1260 

.0560 

.0820 

.0320 

D  = 

■  diag 

(d2 

d2  1) 

18 

18 

24 

.0060 

.0095 

.0150 

.0045 

15 

15 

30 

.0010 

.0120 

.0215 

.0020 

16 

22 

22 

.0070 

.0105 

.0180 

.0050 

12 

24 

24 

.0030 

.0090 

.0180 

.0040 

D  ■ 

-  diag 

(d2 

d2  1/d2) 

18 

18 

24 

.0180 

.0205 

.0270 

.0085 

15 

15 

30 

.0205 

.0220 

.0325 

.0075 

16 

22 

22 

.0290 

.0120 

.0200 

.0040 

12 

24 

24 

.0735 

.0215 

.0280 

.0055 

Note.   When  n1  =  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n,  ¥■   n2  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
ts  are  outside  the  interval  of  a  i  2ya(l-a)/2000 . 
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Table  65 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  10.  p  -   6,  d  -  3,  and  a  =  .01 


N/p  n1   nj  n3     Pillai-    Johansen's    James'      James' 
Bartlett     test       1st  order    2nd  order 
trace  test        test 


D  =  d2I 

18   18   24  .0040 

15  15   30  .0015 

16  22   22  .0050 
12   24   24  .0015 

D  -  diag(d2  d2  d2  1  1  1) 


.0115 

.0160 
.0250 

.0030 

.0185 

.0085 

.0125 

.0215 

.0045 

.0110 

.0180 

.0045 

18  18  24  .0130  .0115  .0190  .0065 

15  15  30  .0130  .0145  .0260  -0035 

16  22  22  .0105  .0180  .0255  .0050 
12  24  24  .0120  .0275  .0405  .0095 

D  =  diag(d2  d2  d2  1/d2  1/d2  1/d2) 

18  18  24  .0110  .0120  .0190  .0025 

15  15  30  .0090  .0165  .0265  .0055 

16  22  22  .0095  .0145  .0225  .0035 
12  24  24  .0100  .0215  .0275  .0060 


Note.   When  a,  =  r^  *  n3,  the  form  of  contamination  is  I:I:D;  when 
n,  ^  ru  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
fs  are  outside  the  interval  of  a  i  27a(l-a)/2000 . 
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Table  66 

Actual  Tvoe 

I  Error 

Rates 

for 

Four  Multivariate  Tests 

01 

Negative 

Cor 

dition. 

N/p  = 

10, 

p  -  6.  d 

■ 

3 .  and  a  = 

N/P  n, 

a. 

n3 

E 

Pillai- 
artlett 

Johansen 
test 

s 

James ' 
1st  order 

James ' 
2nd  order 

trace 

test 

test 

D  -  d2I 

18 

18 

24 

.0385 

.0215 

.0335 

.0065 

15 

15 

30 

.1195 

.0280 

.0545 

.0040 

16 

22 

22 

.0845 

.0275 

.0405 

.0130 

12 

24 

24 

.2210 

.0605 

.0915 

.0220 

.0145 
.0220 
.0190 
.0275 


D   -    diag(d2   d2   d2  1    1    1) 

18      18      24  .0170 

15  15      30  .0420 

16  22      22  .0405 
12      24      24  .0745 

D  =   diag(d2  d2  d2  1/d2   1/d2   1/d2) 

18   18   24  .0105       .0115 

15  15   30  .0215 

16  22   22 
12   24   24 


,0340 


0625 


.0225 


,0400 


.0290 


.0430 


0155 


.0030 
.0050 
.0075 
.0120 

.0045 


.0210 

.0305 

.0070 

.0150 

.0225 

.0060 

.0185 

.0265 

.0085 

Note.   When  n,  -  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *  r^  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  a  ±  2ya(l-a)/2000 . 


125 


Table  67 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =15.  p  -  3,  d  -  VT3.  and  a  -  .  01 


ni 

"a 

n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  - 

d2I 

13 

13 

19 

.0060 

.0080 

.0120 

.0065 

11 

11 

23 

.0030 

.0090 

.0145 

.0070 

13 

16 

16 

.0055 

.0095 

.0110 

.0085 

9 

18 

18 

.0055 

.0075 

.0115 

.0040 

D  = 

diag 

(d2 

d2  1) 

13 

13 

19 

.0035 

.0070 

.0095 

.0065 

11 

11 

23 

.0065 

.0095 

.0125 

.0070 

13 

16 

16 

.0075 

.0085 

.0130 

.0075 

9 

18 

18 

.0085 

.0115 

.0180 

.0080 

D  - 

diag 

(d2 

d2  1/d2) 

13 

13 

19 

.0700 

.0125 

.0140 

.0085 

11 

11 

23 

.0065 

.0130 

.0190 

.0100 

13 

16 

16 

.0095 

.0110 

.0145 

.0075 

9 

18 

18 

.0090 

.0160 

.0210 

.0080 

Note .   When  n,  -  r^  *   n3 ,  the  form  of  contamination  is  I:I:D;  when 
n  yt   n     -  n   the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  a  i  27a(l-a)/2000 . 
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Table  68 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =  15.  p  =  3,  d  =  71 . 5  .  and  a  =  .01 


ru  n3     Pillai-    Johansen's    James'      James' 
Bartlett     test       1st  order    2nd  order 
trace  test        test 


D  = 

d2I 

13 

13 

19 

.0175 

.0130 

.0200 

.0075 

11 

11 

23 

.0125 

.0100 

.0180 

.0065 

13 

16 

16 

.0120 

.0110 

.0175 

.0090 

9 

18 

18 

.0165 

.0130 

.0185 

.0080 

D  = 

diag 

(d2 

d2  1) 

13 

13 

19 

.0080 

.0140 

.0180 

.0100 

11 

11 

23 

.0030 

.0130 

.0175 

.0095 

13 

16 

16 

.0085 

.0120 

.0160 

.0070 

9 

18 

18 

.0075 

.0140 

.0180 

.0110 

D  = 

diag 

(d2 

d2  1/d2) 

13 

13 

19 

.0135 

.0175 

.0230 

.0105 

11 

11 

23 

.0080 

.0125 

.0155 

.0085 

13 

16 

16 

.0165 

.0105 

.0150 

.0080 

9 

18 

18 

.0350 

.0100 

.0150 

.0700 

Note.   When  n,  =  r^  *  n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *  n2  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  a   ±  27q(1-q)/2O0O . 


Table  69 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  15,  p  -  6,  d  -  JTTS .  and  a  =  .01 
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fl, 


Pillai- 
iartlett 
trace 


Johansen' s 
test 


D   =   dzI 

27  27  36 

22  22  46 

26  32  32 

18  36  36 

D  -   diag(d2  d2  d2  1    1    1) 

13  13  19  .0115 

11  11  23  .0135 

13  16  16  .0090 

9  18  18  .0145 

D  =   diag(d2  d2  d2  1/d2   1/d2   1/d2) 

13  13  19  .0085       .0095 

11  11  23  .0105       .0130 

13  16  16  .0075        .0095 

9  18  18  .0095        .0145 


.0120 
.0110 
.0115 
.0140 


James ' 
1st  order 
test 


.0160 


0170 


.0145 


.0210 


James ' 

2nd  order 

test 


.0065 

.0090 

.0130 

.0065 

.0050 

.0120 

.0180 

.0075 

.0065 

.0100 

.0140 

.0040 

.0040 

.0115 

.0155 

.0055 

0090 
.0075 
.0100 
.0090 


.0140 

.0065 

.0220 

.0095 

.0140 

.0055 

.0235 

.0090 

Note .   When  n1  =  r^  *   n3 ,  the  form  of  contamination  is  I:I:D;  when 
n1  *   r^  =  n3,  the  form  of  contamination  is  I :  V:D.      The  underlined 
rs  are  outside  the  interval  of  a  i  27q(1-q)/2000 . 
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Table  70 

Actual  TvDe  I  Error  Rates 

for 

Four  Multivariate  Tests: 

Negative 

Condition.  N/p  = 

15, 

_e_ 

-  6.  d  - 

J\ .  5  ,  and  a  = 

.01 

ni 

°2   "3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

J  anie  s  ' 
1st  order 

test 

James ' 
2nd  order 
test 

D  =  d2I 

27 

27   36 

.0075 

.0090 

.0100 

.0045 

22 

22   46 

.0200 

.0165 

.0215 

.0070 

26 

32   32 

.0125 

.0105 

.0150 

.0050 

18 

36   36 

.0230 

.0170 

.0260 

.0095 

D  -  diag 

(d2  d2  d2 

1  1  1) 

13 

13   19 

.0260 

.0150 

.0205 

.0125 

11 

11   23 

.0420 

.0175 

.0240 

.0080 

13 

16   16 

.0320 

.0120 

.0140 

.0095 

9 

18   18 

.0835 

.0195 

.0280 

.0120 

D  -  diag 

(d2  d2  d2 

1/d2  1/d2 

1/d2) 

13 

13   19 

.0175 

.0075 

.0135 

.0045 

11 

11   23 

.0300 

.0165 

.0250 

.0115 

13 

16   16 

.0370 

.0110 

.0170 

.0090 

9 

18   18 

.084  5 

.0155 

.0200 

.0105 

Note .   When  n1  =  r^  *   n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *   n,  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  a  i  2ja(l-a) /20QQ . 
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Table  71 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  15,  p  =  3,  d  =  3.  and  a   =    .01 


n,   rig   n3     Pillai-     Johansen's     James'        James' 
Bartlett     test       1st  order    2nd  order 
trace  test        test 


D  - 

d2I 

13 

13 

19 

.0035 

.0085 

.0110 

.0070 

11 

11 

23 

.0015 

.0095 

.0180 

.0070 

13 

16 

16 

.0105 

.0140 

.0200 

.0085 

9 

18 

18 

.0030 

.0145 

.0195 

.0115 

D  - 

diag 

(d2 

d2  1) 

27 

27 

36 

.0055 

.0100 

.0120 

.0060 

22 

22 

46 

.0045 

.0135 

.0180 

.0070 

26 

32 

32 

.0105 

.0175 

.0220 

.0115 

18 

36 

36 

.0080 

.0135 

.0190 

.0045 

D  - 

diag 

(d2 

d2  1/d2) 

27 

27 

36 

.0075 

.0100 

.0160 

.0055 

22 

22 

46 

.0090 

.0120 

.0175 

.0065 

2  6 

32 

32 

.0115 

.0100 

.0120 

.0065 

18 

36 

36 

.0110 

.0145 

.0215 

.0050 

Note .   When  n.    =  r^  *   n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *   r^  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  q  ±  27q(1-q)/2000 . 
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Table  72 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition.  N/p  -  15.  p  -  3,  d  -   3,  and  a   =  .01 


n,  n3     Pillai-    Johansen's    James'      James' 
Bartlett     test       1st  order   2nd  order 
trace  test        test 


D  =  d2I 

13  13  19     ,0330       .0100        .0190       .0055 

11  11  23     .0805       .0160        .0250       .0125 

13  16  16     .0535       .0210        .0265       .0140 

9  18  18     .0140       .0230        .0340       .0140 

D  =   diag(d2  d2   1) 

27  27  36 

22  22  46 

26  32  32 
18  36  36 

D   =   diag(d2  d2    1/d2) 

27  27  36  .0105 
22  22  46  .0155 
26  32  32  .0315 
18  36  36     .0720 


.0050 

.0100 

.0120 

.0065 

.0015 

.0135 

.0170 

.0075 

.0100 

.0105 

.0175 

.0065 

.0080 

.0140 

.0145 

.0080 

.0090 

.0135 
.0155 

.0045 

.0095 

.0050 

.0125 

.0160 

.0070 

.0145 

.0195 

.0080 

Note.   When  n,  =  ix,  *  n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *  r^  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  q  ±  2ja( l-a)/2000 . 
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Table  73 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  15.  p  =  6.  d  -  3,  and  a  -  .01 


n,   ru  n3     Pillai-    Johansen's    James'       James' 
Bartlett     test       1st  order    2nd  order 
trace  test         test 


D  -  d2I 

27  27  36 

22  22  46 

26  32  32 
18  36  36 

D  =  diag(d2  d2  d2  1  1  1) 

27  27  36  .0110 
22  22  46  .0185 

26  32  32  .0100 
18  36  36  .0115 

D  =   diag(d2  d2   d2  1/d2   1/d2   1/d2) 

27  27  36  .0135  .0115 
22  22  46  .0090  .0130 
26  32  32  .0120  .0120 
18  36  36  .0110       .0135 


.0060 

.0090 

.0100 

.0055 

.0005 

.0085 

.0115 

.0060 

.0095 

.0115 

.0160 

.0075 

.0020 

.0105 

.0125 

.0060 

.0120 

.0155 

.0070 

.0250 

.0280 

.0100 

.0085 

.0100 

.0065 

.0170 

.0230 

.0080 

.0145 

.0085 

.0210 

.0065 

.0145 

.0075 

.0195 

.0055 

Note.   When  n,  -  r^  *  n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *  r^   -  n3,  the  form  of  contamination  is  I : D : D .   The  underlined 
ts  are  outside  the  interval  of  a  ±  2ya(l-a)/2000 . 
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Table  74 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =  15,  p  =  6,  d  =  3.  and  a  =  .01 


a.   rio   n3     Pillai-     Johansen's     James'       James' 
Bartlett     test       1st  order   2nd  order 
trace  test        test 


.0385 

.0175 
.0095 

.0135 
.0205 

.0050 

.1120 

.0030 

.0630 

.0115 

.0185 

.0060 

.2080 

.0185 

.0275 

.0075 

D  -  d2I 

27  27  36 

22  22  46 

26  32  32 
18  36  36 

D  -  diag(d2  d2  d2  1  1  1) 

27  27  36  .0185 
22  22  46  .0460 

26  32  32  .0330 
18  36  36  .0910 

D  =  diag(d2  d2  d2  1/d2  1/d2  1/d2) 

27  27  36  .0195  .0120  .0160  .0070 
22  22  46  .0220  .0155  .0205  .0085 
26  32  32  .0340  .0140  .0200  .0045 
18  36  36  .0780        .0165         .0205        .0095 


.0125 

.0175 

.0070 

.0125 

.0205 

.0065 

.0130 

.0170 

.0090 

.0165 

.0235 

.0040 

Note .   When  a,  =  n^  *  n3,  the  form  of  contamination  is  I:I:D;  when 
n1  r1   r^  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  q  i  2Ja(l-a) /2 000 . 
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Table  75 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition.  N/p  =  20 .  p  -  3 .  d  =  J\ .  5 .  and  a  - 


01 


r^ 


Pillai- 
iartlett 
trace 


D  -  dzI 

18  18  24 

15  15  30 

16  22  22 
12  24  24 

D  -   diag(d2   d2   1) 

18  18  24 

15  15  30 

16  22  22 
12  24  24 

D  -   diag(d2  d2   1/d2) 

18  18  24     .0100 

15  15  30     .0085 

16  22  22     .0090 
12  24  24     .0085 


Johansen' s 
test 


James ' 

1st  order 

test 


James ' 
2nd  order 

test 


.0060 

.0080 

.0105 

.0080 

.0025 

.0075 

.0095 

.0060 

.0050 

.0065 

.0085 

.0055 

.0085 

.0095 

.0140 

.0080 

.0095 

.0135 

.0175 

.0105 

.0050 

.0125 

.0155 

.0090 

.0070 

.0085 

.0105 

.0060 

.0050 

.0075 

.0100 

.0065 

0125 

.0155 

.0100 

0085 

.0115 

.0060 

0100 

.0155 

.0085 

0120 

.0145 

.0100 

Note .   When  n,  =  r^  *  n3,  the  form  of  contamination  is  I:I:D;  when 
a,  *   rij  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  a   ±  2ya(l-a)/2000 . 


Table  76 


134 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 


"l 

a. 

n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  - 

d2I 

18 

18 

24 

.0120 

.0090 

.0110 

.0080 

15 

15 

30 

.0190 

.0110 

.0170 

.0080 

16 

22 

22 

.0120 

.0120 

.0150 

.0095 

12 

24 

24 

.0140 

.0110 

.0165 

.0090 

D  = 

diag 

(d2 

d2  1) 

18 

18 

24 

.0040 

.0080 

.0100 

.0055 

15 

15 

30 

.0015 

.0130 

.0160 

.0100 

16 

22 

22 

.0060 

.0075 

.0105 

.0060 

12 

24 

24 

.0040 

.0110 

.0155 

.0105 

D  - 

diag 

(d2 

d2  1/d2) 

18 

18 

24 

.0100 

.0115 

.0145 

.0085 

15 

15 

30 

.0145 

.0135 

.0165 

.0115 

16 

22 

22 

.0260 

.0135 

.0150 

.0125 

12 

24 

24 

.0340 

.0140 

.0170 

.0100 

Note .   When  n,  =  r^  *  n3,  the  form  of  contamination  is  I : I : D ;  when 
n,  *  r^  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  a  i  27a(l-a)/2000 . 
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Table  77 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  20,  p  =  6.  d  -  ./T75 .  and  a   =  .01 


n,   n3     Pillai-     Johansen's     James'        James' 
Bartlett     test       1st  order    2nd  order 


trace 

test 

test 

D  ■ 

•  d2I 

36 

36 

48 

.0060 

.0085 

.0105 

.0075 

30 

30 

60 

.0075 

.0135 

.0150 

.0105 

32 

44 

44 

.0080 

.0085 

.0095 

.0060 

24 

48 

48 

.0055 

.0110 

.0125 

.0090 

D  -  diag(d2  d2  d2  1  1  1) 

18   18   24  .0100 

15  15      30  .0115 

16  22      22  .0115 
12      24      24  .0145 

D  =  diag(d2  d2  d2  1/d2   1/d2   1/d2) 

18   18   24  .0135       .0110 

15  15   30  .0100       .0070 

16  22   22  .0140       .0110 
12   24   24  .0085       .0110 


.0120 

.0090 

.0075 

.0135 

.0200 

.0090 

.0115 

.0140 

.0090 

.0095 

.0145 

.0085 

.0135 

.0100 

.0090 

.0050 

.0140 

.0090 

.0155 

.0095 

Note.   When  n,  =  tl,  *  n3,  the  form  of  contamination  is  I :  I :  D ;  when 
n  ft   r^  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  a  i  2ja{ l-a)/2000 . 
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Table  78 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =  20.  p  =  6.  d  -  71.5.  and  a   -  .01 


r^  n3     Pillai-    Johansen's    James'       James' 
Bartlett     test       1st  order    2nd  order 
trace  test         test 


D  - 

d2I 

36 

36 

48 

.0090 

.0090 

.0120 

.0060 

30 

30 

60 

.0170 

.0095 

.0120 

.0060 

32 

44 

44 

.0075 

.0050 

.0060 

.0045 

24 

48 

48 

.0215 

.0140 

.0180 

.0085 

D  = 

diag 

(d2 

d2  d2 

1  1  1) 

18 

18 

24 

.0115 

.0060 

.0075 

.0055 

15 

15 

30 

.0420 

.0175 

.0225 

.0135 

16 

22 

22 

.0515 

.0160 

.0210 

.0110 

12 

24 

24 

.0935 

.0175 

.0210 

.0120 

D  = 

diag 

(d2 

d2  d2 

1/d2  1/d2 

1/d2) 

18 

18 

24 

.0220 

.0100 

.0140 

.0080 

15 

15 

30 

.0345 

.0135 

.0160 

.0085 

16 

22 

22 

.0425 

.0110 

.0130 

.0085 

12 

24 

24 

.0860 

.0175 

.0205 

.0125 

Note .   When  n,  =  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n1  *   nj  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  a  i  2ya(l-a)/2000 . 
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Table  79 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =20,  p  =  3,  d  =  3,  and  a   =  . 01 


n, 

a. 

n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  = 

d2I 

18 

18 

24 

.0095 

.0060 

.0095 

.0045 

15 

15 

30 

.0020 

.0125 

.0155 

.0110 

16 

22 

22 

.0095 

.0115 

.0160 

.0095 

12 

24 

24 

.0050 

.0120 

.0155 

.0100 

D  - 

diag 

(d2 

d2  1) 

36 

36 

48 

.0085 

.0090 

.0100 

.0075 

30 

30 

60 

.0085 

.0145 

.0170 

.0110 

32 

44 

44 

.0110 

.0145 

.0165 

.0115 

24 

48 

48 

.0055 

.0075 

.0100 

.0055 

D  - 

diag 

(d2 

d2  1/d2) 

36 

36 

48 

.0095 

.0110 

.0125 

.0095 

30 

30 

60 

.0100 

.0085 

.0125 

.0055 

32 

44 

44 

.0120 

.0110 

.0120 

.0080 

24 

48 

48 

.0135 

.0110 

.0140 

.0075 

Note.   When  n,  -  r^  *  n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *   r^  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
ts  are  outside  the  interval  of  a   ±  2ja( l-a)/2000 . 
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Table  80 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition.  N/p  =  20,  p  -  3 .  d  =  3.  and  a  ;  .01 


ru   n3     Pillai-     Johansen's     James'        James' 
Bartlett     test       1st  order    2nd  order 
trace  test         test 


D  -  d2I 

18 

18   24 

.0270 

.0085 

.0130 

.0070 

15 

15   30 

.0645 

.0145 

.0215 

.0105 

16 

22   22 

.0670 

.0145 

.0190 

.0110 

12 

24   24 

.1475 

.0190 

.0260 

.0140 

D  =  diag 

(d2  d2  1) 

36 

36   48 

.0060 

.0115 

.0145 

.0090 

30 

^0   60 

.0020 

.0125 

.0140 

.0110 

32 

.   44 

.0090 

.0095 

.0100 

.0060 

24 

]      48 

.0020 

.0055 

.0075 

.0035 

D  =  diag 

(   d2  1/d2) 

36 

i   48 

.0145 

.0065 

.0090 

.0045 

30 

i   60 

.0195 

.0120 

.0155 

.0100 

32 

44 

.0450 

.0130 

.0150 

.0090 

24 

48 

.0810 

.0090 

.0105 

.0055 

Note.   Whi   n,  =  n0  ¥■   n, ,  the  form  of  contamination  is  I:I:D;  when 


n2 


the  form  of  contamination  is  I:D:D.   The  underlined 


re  oul  ide  the  interval  of  a   ±    2ja(l-a) /2000 . 
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Table  81 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition.  N/p  =  20,  p  =  6.  d  -  3.  and  a  =  .01 


r^  n3     Pillai-    Johansen's    James'      James' 
Bartlett     test       1st  order    2nd  order 
trace  test        test 


.0060 

.0100 

.0120 

.0075 

.0010 

.0085 

.0090 

.0050 

.0050 

.0100 

.0120 

.0055 

.0040 

.0095 

.0130 

.0060 

D  -=  d2I 

36  36  48 

30  30  60 

32  44  44 

24  48  48 

D  -  diag(d2  d2  d2  1  1  1) 

36  36  48  .0100 

30  30  60  .0135 

32  44  44  .0115 

24  48  48  .0185 

D  =   diag(d2  d2  d2  1/d2   1/d2   1/d2) 

36  36  48  .0110       .0105        .0165       .0060 

30  30  60  .0105       .0125        .0160       .0060 

32  44  44  .0120       .0115        .0130       .0095 

24  48  48  .0110        .0130         .0175        .0060 


.0100 

.0120 

.0080 

.0100 

.0135 

.0075 

.0100 

.0115 

.0065 

.0180 

.0230 

.0100 

Note.   When  n,  =  r^  *   n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *  r^  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
ts   are  outside  the  interval  of  a  i  2ya(l-a)/2000 . 


Table  82 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition.  N/p  =  20.  p  -  6,  d  -  3.  and  a   =  .01 
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D  -   diag(d2  d2  d2   1/d2   1/d2   1/d2) 

36  36  48  .0105  .0100 
30  30  60  .0185  .0080 
32   44   44     .0410       .0125 

.0125 


24  48  ki 


,0765 


°1 

a,  n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

J  ame  s ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  - 

•  d2I 

36 

36   48 

.0415 

.0105 

.0170 

.0080 

30 

30   60 

.1110 

.0175 

.0205 

.0080 

32 

44   44 

.0900 

.0130 

.0170 

.0090 

24 

48   48 

.2170 

.0220 

.0260 

.0065 

D  = 

=  diag 

(d2  d2  d2 

1  1  1) 

36 

36   48 

.0200 

.0110 

.0145 

.0065 

30 

30   60 

.0360 

.0135 

.0170 

.0070 

32 

44   44 

.0415 

.0145 

.0180 

.0100 

24 

48   48 

.1035 

.0175 

.0215 

.0095 

.0125 
.0105 
.0135 
.0140 


.0005 
.0055 
.0090 
.0080 


Note .   When  n1  -  n2  *  n3,  the  form  of  contamination  is  I:I:D;  when 
n1  *  rir,   -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  of  a  i  2ya(l-a)/2000 . 


APPENDIX  C 

RESULTS  OF  THE  SIMULATED  EXPERIMENTS 

WHEN  SAMPLE  SIZES  ARE  UNEQUAL  AND  a   =    . 10 


Table  83 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 


N/p  n, 

n2 

n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

J  ame  s ' 

1st  order 

test 

James ' 
2nd  order 
test 

D  =  d2I 

9 

9 

12 

.0845 

.1020 

.1160 

.0760 

7 

7 

16 

.0540 

.1275 

.1490 

.0915 

8 

11 

11 

.0770 

.1055 

.1245 

.0770 

6 

12 

12 

.0815 

.1330 

.1535 

.0980 

D  =  diag 

(d2 

d2  1) 

9 

9 

12 

.0760 

.1035 

.1190 

.0750 

7 

7 

16 

.0685 

.1275 

.1470 

.0890 

8 

11 

11 

.0860 

.1195 

.1405 

.0880 

6 

12 

12 

.0845 

.1415 

.1575 

.1035 

D  -  diag 

(d2 

d2  1/d2) 

9 

9 

12 

.0865 

.1145 

.1310 

.0855 

7 

7 

16 

.0740 

.1390 

.1550 

.0950 

8 

11 

11 

.1015 

.1120 

.1295 

.0865 

6 

12 

12 

.0900 

.1365 

.1530 

.0990 

Note.   When  n,  -  n^  *   n3>  the  form  of  contamination  is  I:I:D;  when 
n  ^  n_  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
fs  are  outside  the  interval  a  ±  27a(l-Q)/2000 . 
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Table  84 


Negative 

Con 

dition , 

N/d  =  10. 

p  -  3.  d  - 

7l . 5 ,  and  a 

=  .10 

N/p  n. 

°2 

n3 

B 

Pillai- 

artlet 

trace 

Johansen' s 
test 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  =  d2I 

9 

9 

12 

.1005 

.1085 

.1270 

.0835 

7 

7 

16 

.1420 

.1500 

.1740 

.1015 

8 

11 

11 

.1110 

.1215 

.1420 

.0890 

6 

12 

12 

.1230 

.1465 

.1675 

.1010 

D  =  diag 

(d2 

d2  1) 

9 

9 

12 

.1015 

.1170 

.1340 

.0865 

7 

7 

16 

.1125 

.1310 

.1570 

.0895 

8 

11 

11 

.1075 

.1280 

.1485 

.0985 

6 

12 

12 

.1335 

.1465 

.1755 

.1085 

D  =  diag 

(d2 

d2  1/d2) 

9 

9 

12 

.0985 

.1190 

.1345 

.0930 

7 

7 

16 

.1090 

.1305 

.1570 

.0835 

8 

11 

11 

.1095 

.1185 

.1405 

.0855 

6 

12 

12 

.1145 

.1455 

.1655 

.1045 

Note .   When  n,  =  i^  *  n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *  n_  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
ts   are  outside  the  interval  a   ±   27q(1-q)/2000 . 
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Table  85 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  -  10 .  p  -  6 .  d  -  ./l .  5 .  and  a  -  .  10 


N/p  n,   nj  n3     Pillai-    Johansen's    James'      James' 
Bartlett     test       1st  order    2nd  order 
trace  test        test 


D  -  dzI 

18  18  24  .0815       .1220        .1375       -0680 

15  15  30  ,0605  .1345        .1500       .0665 

16  22  22  .0830  .1340        .1195       -0680 
12  24  24  .0645       .1295        .1470       .0675 

D  -  diag  (d2  d2  d2  1  1  1) 

18  18  24  .0905 

15  15  30  .0785 

16  22  22  .0885 
12  24  24  .0810 

D  -  diag  (d2  d2  d2  1/d2  1/d2  1/d2) 

18  18  24 

15  15  30 

16  22  22 
12  24  24 


.1155 

.1290 

.0730 

.1340 

.1550 

.0845 

.1230 

.1350 

.0750 

.1390 

.1600 

.0685 

.0995 

.1275 

.1405 

.0820 

.1025 

.1385 

.1575 

.0685 

.0905 

.1180 

.1360 

.0710 

.1085 

.1425 

.1635 

.0770 

Note.   When  n,  =  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *   r^  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
ts  are  outside  the  interval  a  ±  2ja( l-a)/2000 . 


Table  86 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =  10.  p  =  6.  d  =  713,  and  a  -  .10 
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N/p      n,      r^      n3 


Pillai- 
Jartlett 

trace 


Johansen' s 
test 


James ' 
1st  order 
test 


James ' 
2nd  order 
test 


D  -  d'l 

18  18  24  .1120 

15  15  30  .1395 

16  22  22  .1190 
12  24  24  .1500 

D  -  diag  (d2  d2  d2  1  1  1) 

18  18  24  .1080 

15  15  30  .1140 

16  22  22  .1140 
12  24  24  .1280 

D  =  diag  (d2  d2  d2  1/d2   1/d2   1/d2) 

18  18  24  .0970       .1245 

15  15  30  .0880 

16  22  22  .0945 
12  24  24  .1015 


1250 


1225 


1245 


1670 


1260 


1395 


1260 


1465 


.1340 
.1235 
.1400 


1410 


1485 


1415 


1905 


1450 


1610 


1390 


1635 


1425 


1505 


1380 


1570 


.0625 


.0625 


0765 


.0880 


.0680 


.0690 


.0760 


.0840 


0775 


.0675 


.0770 


.0760 


Note .   When  n,  -  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n1  r*   n^  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a  ±  2ja{ l-a)/2000 . 
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Table  87 

Rates  for  Four  Multivariate  Tests: 


Positive 

Cor 

dition. 

N/p  =  10, 

p  =  3.  d  - 

3 ,  and  a  - 

.10 

N/p  n, 

n. 

n3 

Pillai- 

Johansen' s 

James ' 

James ' 

B 

artlett 

test 

1st  order 

2nd  order 

trace 

test 

test 

D  =  d2I 

9 

9 

12 

.0750 

.1110 

.1255 

.0795 

7 

7 

16 

.0155 

.1180 

.1335 

.0930 

8 

11 

11 

.0785 

.1195 

.1360 

.0905 

6 

12 

12 

.0485 

.1120 

.1280 

.0855 

D  -  diag 

(d2 

d2  1) 

9 

9 

12 

.0845 

.0195 

.1255 

.0855 

7 

7 

16 

.0315 

.1125 

.1340 

.0810 

8 

11 

11 

.0775 

.1235 

.1355 

.0935 

6 

12 

12 

.0610 

.1090 

.1290 

.0825 

D  -  diag 

(d2 

d2  1/d2) 

9 

9 

12 

.0965 

.1105 

.1275 

.0875 

7 

7 

16 

.0855 

.1385 

.1620 

.0905 

8 

11 

11 

.1265 

.1170 

.1330 

.0880 

6 

12 

12 

.1420 

.1195 

.1375 

.0880 

Note.   When  n,  =  r^  t*   n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *  r^  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a   ±  2ya(l-a)/2000 . 
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Table  88 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =  10,  p  -  3.  d  -  3.  and  a   =  .10 


N/p   n,   r^  n3     Pillai-     Johansen's     James'       James' 
Bartlett     test       1st  order   2nd  order 
trace  test        test 


D  - 

d2I 

9 

9 

12 

.1840 

.1530 

.1790 

.1080 

7 

7 

16 

.3500 

.1745 

.2260 

.1100 

8 

11 

11 

.2735 

.1545 

.1755 

.1050 

6 

12 

12 

,4015 

.1890 

.2170 

.1375 

D  - 

diag 

(d2 

d2  1) 

9 

9 

12 

.1530 

.1420 

.1655 

.1020 

7 

7 

16 

.2500 

.1570 

.1870 

.0955 

8 

11 

11 

.2180 

.1425 

.1670 

.1015 

6 

12 

12 

.2975 

.1685 

.1970 

.1155 

D  - 

diag 

(d2 

d2  1/d2) 

9 

9 

12 

.1350 

.1305 

.1460 

.0935 

7 

7 

16 

.2045 

.1540 

.1840 

.1005 

8 

11 

11 

.1880 

.1170 

.1355 

.0850 

6 

12 

12 

.2595 

.1405 

.1560 

.0995 

Note .   When  n,  =  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *   r^  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a  ±  27a(l-a)/2000 . 
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Table  89 


Positive 

Conditior 

.  N/p  =  10. 

P  =  6, 

d  = 

3 .  and  a  - 

.10 

N/p   n, 

l\ 

n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  -  d2I 

18 

18 

24 

.0680 

.1215 

.1295 

.0715 

15 

15 

30 

.0120 

.1285 

.1445 

.0825 

16 

22 

22 

.0700 

.1380 

.1510 

.0860 

12 

24 

24 

.0320 

.1180 

.1315 

.0735 

D  =  diag 

(d2 

d2  d2 

1  1  1) 

18 

18 

24 

.0740 

.1150 

.1325 

.0675 

15 

15 

30 

.0425 

.1315 

.1510 

.0740 

16 

22 

22 

.0695 

.1185 

.1325 

.0680 

12 

24 

24 

.0575 

.1370 

.1520 

.0835 

D  -  diag  (d2  d2  d2  1/d2  1/d2  1/d2) 

18   18  24     .1155       .1305  .1485  .0805 

15  15  30     .1215       .1410  .1635  .0740 

16  22  22     .1695       .1190  .1365  .0730 
12   24  24     .2495       .1340  .1550  .0765 


Note.   When  n,  -  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n.  *   rij  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
ts  are  outside  the  interval  a  ±  2,/a(l-a)/2000 . 
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Table  90 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =10,  p-6,  d-3.  and  a  =  , 10 


N/p   n1   ru   n3     Pillai-     Johansen's     James'       James' 
Bartlett     test       1st  order   2nd  order 
trace  test        test 


D  ■ 

■  d2I 

18 

18 

24 

.2105 

.1615 

.1865 

.0745 

15 

15 

30 

.3840 

.1755 

.2135 

.0735 

16 

22 

22 

.3005 

.1555 

.1775 

.0815 

12 

24 

24 

.4490 

.2120 

.2310 

.1240 

D  ■ 

•  diag 

(d2 

d2  d2 

1  1  1) 

18 

18 

24 

.1345 

.1255 

.1440 

.0710 

15 

15 

30 

.2280 

.1685 

.1985 

.0800 

16 

22 

22 

.2070 

.1320 

.1435 

.0755 

12 

24 

24 

.2865 

.1465 

.1760 

.0775 

D  =  diag  (d2  d2  d2  1/d2  1/d2  1/d2) 

18   18  24     .0955       .1105  .1255  -0655 

15  15  30     .1255       .1350  .1580  .0645 

16  22  22     .1725       .1255  .1370  .0750 
12   24  24     .2260       .1120  .1275  .0660 


Note .   When  n,  =  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *  r^   =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a   ±    2ja(l-a) /2000 . 
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Table  91 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 

'*   ~~~   '  '  .10 


Positive  Condition,  N/p  =  15  .  p  =  3  .  d  ■=  ./l .  5  .  and  a 


N/p   n, 

°2   n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  =  d2I 

13 

13   19 

.0770 

.1030 

.1155 

.0900 

11 

11   23 

.0545 

.0955 

.1055 

.0800 

13 

16   16 

.0890 

.1020 

.1105 

.0910 

9 

18   18 

.0805 

.1090 

.1195 

.0895 

D  -  diag 

(d2  d2  1) 

13 

13   19 

.0845 

.0970 

.1050 

.0830 

11 

11   23 

.0680 

.0100 

.1095 

.0845 

13 

16   16 

.0855 

.0995 

.1090 

.0860 

9 

18   18 

.0855 

.1135 

.1265 

.0950 

D  -  diag 

(d2  d2  1/d2) 

13 

13   19 

.0910 

.0970 

.1050 

.0795 

11 

11   23 

.0900 

.1155 

.1250 

.0935 

13 

16   16 

.1005 

.1030 

.1155 

.0915 

9 

18   18 

.0940 

.1115 

.1215 

.0880 

Note.   When  n1  -  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n  *   n,  =  n3>  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a  ±  27q(1-q)/2000 . 
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Table  92 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 


Negative  Condition,  N/p  =  15,  p  =  3,  d  •=  J\ .  5 ,  and  a 


10 


N/p      rt,      rig      n3  Pillai-  Johansen's  James'  James' 

Bartlet  test  1st   order        2nd   order 

trace  test  test 


.1165 

.1080 

.1190 

.0955 

.1365 

.1150 

.1305 

.0870 

.1215 

.1115 

.1235 

.0970 

.1225 

.1165 

.1315 

.0885 

D   =   d2I 

13  13  19 

11  11  23 

13  16  16 

9  18  18 

D   =   diag(d2  d2    1) 

13  13  19     .1090       .1085        .1210       .0950 

11  11  23     .1240       .1040        .1175       .0855 

13  16  16    .1120       .1155        .1270       .1020 

9  18  18     .1105       .1130        .1250       .0900 

D  =   diag(d2  d2   1/d2) 

13  13  19 

11  11  23 

13  16  16 

9  18  18 


Note.   When  n,  =  r^  *   n3 ,  the  form  of  contamination  is  I:I:D;  when 
a,  *   n,  -  rig,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a   ±  2ya(l-a)/2O0O . 


.1080 

.1145 

.1225 

.0985 

.1135 

.1185 

.1265 

.0970 

.1115 

.1070 

.1155 

.0965 

.1090 

.1170 

.1235 

.0920 
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Table  93 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  15,  p  =  6.  d  -  ./l .  5 .  and  a  -  .  10 


N/p   a,   r^  n3     Pillai-     Johansen's     James'       James' 
Bartlett     test       1st  order    2nd  order 
trace  test        test 


.0955 

.1035 

.1280 

.0715 

.1170 

.0825 

.1030 

.1140 

.0825 

.1175 

.1265 

.0870 

D  -  d2I 

27  27  36  .0710 

22  22  46  .0625 

26  32  32  .0855 
18  36  36  .0725 

D  -  diag  (d2  d2  d2  1  1  1) 

27  27  36  .0900  .1055  .1165  .0775 
22  22  46  .0690       .1095        .1195       .0765 

26  32  32  .1085  .1225  .1320  .1005 
18  36  36  .0810       .1090        .1220       -0745 

D  =  diag  (d2  d2  d2  1/d2  1/d2  1/d2) 

27  27  36  .0845  .0990  .1065  .0765 
22  22  46  .0955  .1105  .1210  .0745 
26  32  32  .0965  .1060  .1125  .0750 
18  36  36  .1145       .1300        .1395       .0885 


Note.   When  n,  -  r^  *  n3,  the  form  of  contamination  is  I:I:D;  when 
n,  ¥■   n2  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a  ±  2ya(l-a)/2000 . 
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Table  94 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition.  N/p  =  15 .  p  -  6 .  d  =  ./l .  5 .  and  a   -  .  10 


N/p   a,   r^  n3 


Pillai- 
Bartlett 
trace 


Johansen' s 
test 


D  -  d2I 

27  27  36 

22  22  46 

26  32  32 

18  36  36 


D  -  diag  (d2  d2  d2  1  1  1) 

27  27  36  .1095  .1110 

22  22  46  .1380  .1170 

26  32  32  .1090  .1125 
18  36  36  .1175  .1265 

D  -  diag  (d2  d2  d2  1/d2  1/d2  1/d2) 

27  27  36  .0995  .1050 
22  22  46  .0100  .1195 
26  32  32  .0915  .1005 
18  36  36     .0990  .1120 


James ' 
1st  order 
test 


1195 
1315 
.1220 
.1355 

.1110 
.1305 
.1095 
.1210 


James ' 
2nd  order 
test 


.1040 

.0965 

.1085 

.0695 

.1540 

.1215 

.1330 

.0885 

.1205 

.1150 

.1225 

.0855 

.1465 

.1265 

.1380 

.0890 

0850 


,0855 


.0785 


,0885 


.0865 


0825 


0800 


.0765 


Note.   When  n,  =  ru,  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n  _<  r^  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
ts  are  outside  the  interval  a  ±  2ya(l-a)/2000 . 


Table  95 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition.  N/p  =  15,  p  =  3.  d=3,  and  a  =  .  10 
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N/p  n, 

nj 

n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  =  d2I 

13 

13 

19 

.0440 

.1005 

.1060 

.0865 

11 

11 

23 

.0225 

.1110 

.1190 

.0980 

13 

16 

16 

.0895 

.1075 

.1175 

.0935 

9 

18 

18 

.0515 

.1045 

.1100 

.0955 

D  =  diag 

(d2 

d2  1) 

13 

13 

19 

.0665 

.1035 

.1120 

.0915 

11 

11 

23 

.0415 

.1075 

.1140 

.0940 

13 

16 

16 

.0985 

.1150 

.1195 

.0995 

9 

18 

18 

.0645 

.1070 

.1115 

.0935 

D  -  diag 

(d2 

d2  1/d2) 

13 

13 

19 

.0965 

.1225 

.1325 

.1085 

11 

11 

23 

.0835 

.1100 

.1185 

.0915 

13 

16 

16 

.1270 

.1165 

.1220 

.0980 

9 

18 

18 

.1375 

.0930 

.1000 

.0770 

Note .   When  n,  =  r^  *   n3 ,  the  form  of  contamination  is  1:1 :D;  when 
n,  *  n,  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a  ±  2ya(l-a)/2000 . 
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Table  96 


Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =  15.  p=3.  d=3.  and  a  =    .  10 


N/p   n1   rij  n3     Pillai-     Johansen's     James'       James' 
Bartlett     test       1st  order   2nd  order 
trace  test        test 


D  = 

d*I 

13 

13 

19 

.1890 

11 

11 

23 

.3015 

13 

16 

16 

.2205 

9 

18 

18 

.3490 

D  = 

diag 

(d2 

d2  1) 

13 

13 

19 

.1605 

11 

11 

23 

.2310 

13 

16 

16 

.1575 

9 

18 

18 

.2835 

D  - 

diag 

(d2 

d2  1/d2) 

13 

13 

19 

.1280 

11 

11 

23 

.1700 

13 

16 

16 

.1655 

9 

18 

18 

.2735 

.1140  .1285  .0855 

.1365  .1590  .0955 

.1170  .1285  .0995 

.1325  .1480  .1060 

.1225  .1340  .0935 

.1195  .1355  .0865 

.1065  .1120  .0895 

.1280  .1440  .0970 

.1090  .1180  .0895 

.1210  .1375  .0915 

.1085  .1190  .0955 

.1240  .1385  .1000 


Note .   When  n,  -  r^  i*   n3,  the  form  of  contamination  is  I:I:D;  when 
Ui  ^  r^  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a   ±    2ja(l-a) /2000 . 
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Table  97 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition.  N/p  =15,  p  =  6.  d  =  3 ,  and  a   =    .10 


N/p  n,   rt,  n3 


Pillai- 
Bartlett 
trace 


D  =  d^I 

27  27  36 

22  22  46 

26  32  32 

18  36  36 


0670 


Johansen' s 
test 


.1040 
.1090 
.1150 
.1025 


.0070 

.0795 

.0350 

D  =  diag  (d2  d2  d2  1  1  1) 

27   27   36     .0710 

22   22   46     .0335 

26  32   32     .0855 
18   36   36    .0650 

D  =  diag  (d2  d2  d2  1/d2  1/d2  1/d2) 

27  27  36  .1045  .1000 
22  22  46  .1245  .1070 
26  32  32  .1510  .1030 
18   36   36    .2390       .1120 


James ' 
1st  order 
test 


1160 


1175 


1260 


1100 


.0995 

.1085 

.1080 

.1105 

.1035 

.1130 

.1040 

.1120 

1050 
1205 
1120 
1215 


James ' 
2nd  order 

test 


.0785 


.0800 


0800 


.0865 


.0765 


.0805 


.0845 


.0810 


.0680 


.0710 


.0780 


!15 


Note .   When  n,  -  n?  *   n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *   r^  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a   ±  27q(1-q)/2000 . 
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Table  98 

Actual  Type 

T  Er 

ror  Rates 

for 

Four  Multivariate  Tests 

10 

Negative 

Cor 

diti 

on.  N/p  = 

15, 

p  =  6.  d 

= 

3 .  and  a  =  . 

N/p   n, 

r^ 

n3 

Pillai- 

Bartlett 
trace 

Johansen 
test 

s 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  -  d2I 

27 

27 

36 

.1995 

.1210 

.1325 

.0740 

22 

22 

46 

.3925 

.1245 

.1455 

.0615 

26 

32 

32 

.2460 

.1080 

.1185 

.0770 

18 

36 

36 

.4820 

.1285 

.1415 

.0825 

2315 


1680 


3025 


.1130 
.1215 
.1200 
.1180 


D  =  diag  (d2  d2  d2  1  1  1) 

27  27   36  .1410 

22  22   46 

26  32   32 
18  36   36 

D  =   diag  (d2  d2  d2  1/d2   1/d2   1/d2) 

27  27  36  .1045  .1105 
22  22  46  .1285  .1235 
26  32  32  .1490  .1185 
18  36   36  .2245 


.1090 


1220 


1350 


1295 


1315 


1190 


.0845 


0760 


.0865 


.0795 


.0805 


.1335 

.0810 

.1260 

.0950 

.1155 

.0875 

Note .   When  n,  =  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *   rij   =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
fs  are  outside  the  interval  a  ±   27a(l-a)/2000 . 
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Table  99 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 


Positive  Condition.  N/p  =20.  p-  3.  d-  71 .5.  and  a 


10 


N/p      n,      r^     n3 


Pillai- 
Jartlett 

trace 


D  -   d2I 

18  18  24 

15  15  30 

16  22  22 
12  24  24 

D  =   diag(d2  d2   1) 

18  18  24 

15  15  30 

16  22  22 
12  24  24 

D  -  diag(d2  d2  1/d2) 

18  18  24     .0995 

15  15  30  .0900 

16  22  22  .0915 
12  24  24     .0910 


Johansen' s 
test 


James ' 
1st  order 
test 


James ' 

2nd  order 

test 


.0810 

.0835 

.0915 

.0770 

.0735 

.1060 

.1120 

.0940 

.0865 

.1105 

.1150 

.0995 

.0715 

.1030 

.1110 

.0885 

.0920 

.1045 

.1100 

.0960 

.0685 

.1050 

.1090 

.0935 

.0890 

.1040 

.1085 

.0965 

.0720 

.1040 

.1110 

.0905 

.1115 

.1150 

.1035 

.1165 

.1215 

.1030 

.0930 

.1000 

.0860 

.0965 

.1045 

.0865 

Note.   When  n1  -  r^  +   n3,  the  form  of  contamination  is  I:I:D;  when 
n  *   nj  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
?s  are  outside  the  interval  a   ±  27a(l-Q)/2000 . 
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Table  100 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition.  N/p  =  20 ,  p  =  3 .  d  =  ./l .  5 .  and  a  =  ,  10 


N/p   a,   nj   n3     Pillai-     Johansen's     James'       James' 
Bartlet      test       1st  order   2nd  order 
trace  test        test 


D  -  d2I 

18 

18 

24 

.1205 

.1025 

.1135 

.0970 

15 

15 

30 

.1305 

.1005 

.1075 

.0875 

16 

22 

22 

.1200 

.1030 

.1100 

.0945 

12 

24 

24 

.1265 

.1030 

.1100 

.0885 

D  -  diag 

(d2 

d2  1) 

18 

18 

24 

.1125 

.1100 

.1140 

.1010 

15 

15 

30 

.1180 

.1065 

.1120 

.0930 

16 

22 

22 

.1030 

.0990 

.1040 

.0880 

12 

24 

24 

.1160 

.1040 

.1105 

.0925 

D  -  diag 

(d2 

d2  1/d2) 

18 

18 

24 

.1010 

.0980 

.1055 

.0915 

15 

15 

30 

.1095 

.1100 

.1175 

.0970 

16 

22 

22 

.1055 

.1115 

.1175 

.0995 

12 

24 

24 

.1175 

.1110 

.1155 

.0925 

Note .   When  n,  =  r^  *   n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  ¥■   ru  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a  ±  27a(l-a)/2000 . 


Table  101 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition.  N/p  =  20.  p  =  6.  d  -  ./! . 5 .  and  a  -  .10 
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N/p   n,   ix,  n3 


Pillai- 

Bartlett 
trace 


Johansen' s 
test 


D  =  d2I 

36  36  48  .0705 

30  30  60  .0660 

32  44  44  .0725 

24  48  48  .0635 

D  =  diag  (d2  d2  d2  1  1  1) 

36  36  48  .0805 

30  30  60  .0725 

32  44  44  .1025 

24  48  48  .0845 

D  -  diag  (d2  d2  d2  1/d2  1/d2  1/d2) 

36  36  48  .1100       .1140 

30  30  60  .0100       .1145 

32  44  44  .1020       .1000 

24  48  48  .1110       .1160 


James ' 
1st  order 
test 


J  ame  s ' 

2nd  order 

test 


.0880 

.0925 
.1130 

.0785 

.1070 

.0960 

.0940 

.0980 

.0800 

.1005 

.1080 

.0805 

.1005 

.1060 

.0870 

.1040 

.1080 

.0845 

.1100 

.1155 

.0930 

.1090 

.1115 

.0935 

.1190 

.0970 

.1230 

.0935 

.1045 

.0885 

.1240 

.0925 

Note.   When  n,  -  rt,  *   n3 ,  the  form  of  contamination  is  I:I:D;  when 
n  *  n-,  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
ts    are  outside  the  interval  a  ±  27a(l-a)/2000 . 


Table  102 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  -20.  p-6.  d-./1.5.  and  a   -  .  10 
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N/p  n,   rig  n3 


Pillai- 
Bartlett 
trace 


Johansen' s 
test 


D  -  d^I 

36  36  48 

30  30  60 

32  44  44 

24  48  48 

D  -  diag  (d2  d2  d2  1  1  1) 

36  36  48  .0950 

30  30  60  .1250 

32  44  44  .1045 

24  48  48  .1265 

D  -  diag  (d2  d2  d2  1/d2  1/d2  1/d2) 

36  36  48  .0980       .1050 

30  30  60  .1075       .1210 

32  44  44  .1075       .1075 

24  48  48  .1095       .1175 


.0970 
.1055 
.1000 
.1105 


James ' 

1st  order 

test 


1010 
1140 
1050 
1160 


James ' 
2nd  order 
test 


1190 

.0955 
.1105 

.1030 
.1190 

.0785 

.1520 

.0890 

.1115 

.0970 

.1010 

.0760 

.1500 

.1200 

.1260 

.0875 

0825 


,0830 


0845 


,0860 


.1115 

.0895 

.1250 

.1005 

.1135 

.0940 

.1270 

.0955 

Note.   When  n,  =  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n,  *   nj  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a  ±  2,/a(l-a)/2000 . 


Table  103 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =  20.  p  =  3,  d  -=  3,  and  a  =  .10 
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N/p  n. 

n2 

n3 

Pillai- 
Bartlett 
trace 

Johansen 
test 

s     James' 
1st  order 
test 

James ' 
2nd  order 
test 

D  -  d2I 

18 

18 

24 

.0745 

.1020 

.1065 

.0940 

15 

15 

30 

.0250 

.1080 

.1135 

.0995 

16 

22 

22 

.0735 

.1035 

.1085 

.0920 

12 

24 

24 

.0555 

.1015 

.1075 

.0960 

D  -  diag 

(d2 

d2  1) 

18 

18 

24 

.0770 

.0975 

.1015 

.0920 

15 

15 

30 

.0400 

.1015 

.1055 

.0930 

16 

22 

22 

.0850 

.1085 

.1130 

.1010 

12 

24 

24 

.0665 

.1035 

.1065 

.0985 

D  =  diag 

(d2 

d2  1/d2) 

18 

18 

24 

.1020 

.1030 

.1080 

.0940 

15 

15 

30 

.0775 

.1070 

.1135 

.0965 

16 

22 

22 

.1335 

.1020 

.1120 

.0965 

12 

24 

24 

.1485 

.1100 

.1175 

.0990 

Note .   When 

ni  = 

rij  ¥■   n3,  the 

form  of  c 

ontamination  is 

I :  I : D ;  when 

a,  **  n,  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a  ±  2ya(l-a)/2000 . 


Table  104 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Negative  Condition,  N/p  =20.p=3.d=3.  and  a  =  . 10 
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N/p   n, 

tl,  n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  =  d2I 

18 

18   24 

.1740 

.1040 

.1110 

.0900 

15 

15   30 

.2660 

.1040 

.1155 

.0825 

16 

22   22 

.2385 

.1055 

.1145 

.0910 

12 

24   24 

.3745 

.1265 

.1365 

.1035 

D  -  diag 

(d2  d2  1) 

18 

18   24 

.1215 

.0960 

.1035 

.0800 

15 

15   30 

.2125 

.1140 

.1215 

.0945 

16 

22   22 

.2030 

.1135 

.1220 

.1025 

12 

24   24 

.2865 

.1180 

.1295 

.0965 

D  =  diag 

(d2  d2  1/d2) 

18 

18   24 

.1280 

.1090 

.1140 

.0980 

15 

15   30 

.1505 

.1045 

.1100 

.0900 

16 

22   22 

.1675 

.1050 

.1105 

.0985 

12 

24   24 

.2440 

.1030 

.1105 

.0920 

Note.   When  n,  -  r^  *   n3 ,  the  form  of  contamination  is  I:I:D;  when 
n,  *   n,  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
fs  are  outside  the  interval  a  ±  27a(l-a)/2000 . 


Table  105 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 
Positive  Condition,  N/p  =20,  p-6.  d  =  3.  and  a   =  . 10 
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N/p  n, 

a. 

n3 

Pillai- 
Bartlett 
trace 

Johansen 
test 

s    James' 
1st  order 
test 

James ' 
2nd  order 
test 

D  =  d2I 

36 

36 

48 

.0625 

.1030 

.1075 

.0930 

30 

30 

60 

.0095 

.0910 

.0945 

.0795 

32 

44 

44 

.0600 

.1055 

.1090 

.0895 

24 

48 

48 

.0405 

.1090 

.1145 

.0915 

D  =  diag 

(d2 

d2  d2 

1  1  1) 

36 

36 

48 

.0815 

.1090 

.1145 

.0950 

30 

30 

60 

.0390 

.1020 

.1090 

.0845 

32 

44 

44 

.0775 

.1090 

.1135 

.0870 

24 

48 

48 

.0535 

.0915 

.0960 

.0745 

D  =  diag 

(d2 

d2  d2 

1/d2  1/d2 

1/d2) 

36 

36 

48 

.0940 

.0955 

.1025 

.0780 

30 

30 

60 

.1035 

.1145 

.1210 

.0930 

32 

44 

44 

.1805 

.1110 

.1170 

.0955 

24 

48 

48 

.2525 

.1135 

.1200 

.0955 

Note .   When 

n,  -  i 

ij  *   n3 ,  th 

e  form  of  c 

ontamination  is  I:I:D;  when 

n,  *   r^  =  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 
rs  are  outside  the  interval  a  ±   2ja(l-a) /2000 . 
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Table  106 

Actual  Type  I  Error  Rates  for  Four  Multivariate  Tests: 


Negative  Condition,  N/p  =20.  p  =  6.  d  ^  3.  and  a 


10 


N/P  n. 

a. 

n3 

Pillai- 
Bartlett 
trace 

Johansen' s 
test 

James ' 
1st  order 
test 

James ' 
2nd  order 
test 

D  =  d2I 

36 

36 

48 

.1940 

.1115 

.1195 

.0875 

30 

30 

60 

.3530 

.1185 

.1275 

.0890 

32 

44 

44 

.2830 

.1075 

.1155 

.0850 

24 

48 

48 

.4785 

.1160 

.1265 

.0855 

D  =  diag  (d2  d2  d2  1  1  1) 

36  36   48     .1460 

30  30   60     .2175 

32  44   44     .1850 

24  48   48     .3000 

D  -  diag  (d2  d2  d2  1/d2  1/d2  1/d2) 

36  36   48 

30  30   60 

32  44   44 

24  48   48 


.1040 
.1085 
.1055 
.1170 


1140 


1190 


1150 


1215 


0895 
.0790 
.0900 
.0930 


.1015 

.0970 
.1025 

.1030 

.1090 

.0820 

.1265 

.0820 

.1815 

.1120 

.1175 

.0955 

.2485 

.1125 

.1190 

.0870 

Note.   When  n,  =  r^  *   n3,  the  form  of  contamination  is  I:I:D;  when 
n„  *  r^  -  n3,  the  form  of  contamination  is  I:D:D.   The  underlined 


rs  are  outside  the  interval  a  ±  2ja( l-a)/2000 . 
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